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1.0  INTRODUCTION 

This  is  one  of  a  series  of  the  Environmental  Seeurity  Teehnology  Certification  Program  (ESTCP) 
live  site  demonstrations  of  classification  technologies  for  Munitions  Response  (MR).  This 
demonstration  is  designed  to  evaluate  advanced  classification  methodology  at  the  Eormer  Camp 
Ellis  Military  Range  (CEMR),  which  is  a  site  known  to  contain  evidence  of  2.36-inch  practice 
rockets,  hand  grenades,  and  rifle  grenades. 

The  project  team  consists  of  ARC  ADIS  United  States  (U.S.)  Inc.  (ARC  ADIS)  (principal 
investigator)  with  support  by  NAEVA  Geophysics,  Inc.  (NAEVA).  Appendix  A  provides  the 
points  of  contact  information  for  the  ARC  ADIS  team,  as  well  as  the  primary  U.S.  Army  Corps  of 
Engineers  (USACE)  (and  its  subcontractors),  ESTCP  Program  Office,  and  Illinois 
Environmental  Protection  Agency  personnel.  ARCADIS  was  responsible  for  managing  and 
implementation  of  all  field  and  office  tasks  associated  with  the  live  site  demonstration  at  CEMR, 
and  NAEVA  provided  field  and  office  geophysical  support.  Additional  support  was  provided  by 
the  Naval  Research  Eaboratory  (NRL),  Geometries,  and  Science  Applications  International 
Corporation  (SAIC). 

1.1  BACKGROUND 

The  Department  of  Defense  (DoD)  is  responsible  for  investigating  and  cleaning  up  thousands  of 
Munitions  Response  Sites  (MRSs)  comprising  millions  of  acres  that  are  potentially  impacted  by 
military  munitions.  Current  industry-standard  practice  includes  digital  geophysical  mapping 
(DGM)  surveys  and  excavating  a  large  number  of  subsurface  metallic  objects  that  are  not 
munitions  and  explosives  of  concern  (MEC).  These  non-MEC  items  are  not  an  explosive  hazard, 
yet  their  excavation  represents  most  of  DoD’s  MEC  cleanup  costs.  Next  generation 
electromagnetic  induction  (EMI)  sensors  and  advanced  software  algorithms  (e.g.,  in  UX- Analyze 
Advanced)  are  able  to  successfully  classify  geophysical  targets  at  MRSs  into  feature  classes  that 
differentiate  between  MEC,  non-hazardous  munitions  debris  (MD),  and  scrap  metal.  The  ability 
to  classify  targets  will  allow  project  teams  to  focus  intrusive  investigations  on  buried  items  that 
pose  a  potential  explosive  hazard  (e.g.,  MEC),  reduce  the  costs  of  remediation,  and  minimize  the 
impacts  to  the  environment  and  the  public  who  must  evacuate  areas  during  intrusive  operations. 
This  project  evaluated  the  effectiveness  of  conducting  DGM  surveys  with  three  geophysical 
sensors  and  applying  advanced  classification  of  dynamic  and  cued  {i.e.,  static)  survey  data  from 
two  advanced  EMI  sensors. 

During  this  demonstration,  ARCADIS  used  three  geophysical  sensors: 

•  EM61-MK2, 

•  Geometries’  MetalMapper™  (MetalMapper),  and 

•  NRL’s  Time-domain  Electromagnetic  Multi-sensor  Towed  Array  Detection  System 
(TEMTADS)  Man  Portable  (MP)  2x2. 

Dynamic  DGM  data  was  collected  with  all  three  instruments,  and  static  cued  survey  data  was 
collected  with  the  latter  two  instruments,  which  are  both  advanced  EMI  sensors.  The  ability  to 
correctly  classify  anomalies  as  target  of  interest  (TOI)  and  non-TOI  using  the  advanced  EMI 
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sensors  operating  in  statie  or  in  dynamie  mode  (instead  of  the  current  industry  standard 
approach)  offers  the  potential  to  significantly  reduce  the  DoD’s  MEC  cleanup  costs. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  overall  objective  of  the  demonstration  was  to  validate  classification  technology  in  a  live  site 
demonstration  at  the  former  CEMR.  The  overall  objective  of  the  advanced  classification 
demonstration  was  to  accurately  classify  each  of  the  targets  as  one  of  the  following; 

•  TOI  that  has  a  high  likelihood  of  being  unexploded  ordnance  (UXO);  or 

•  Non-TOI  that  is  likely  another  type  of  anomaly  source  such  as  MD,  scrap  metal,  or  some 
other  metallic  unrelated  to  UXO. 

Additional  objectives  for  this  demonstration  included  the  following: 

•  Comparing  the  effectiveness  of  the  three  dynamic  DGM  sensors  in  their  ability  to  detect 
potential  TOI. 

•  Comparing  the  effectiveness  of  the  two  advanced  EMI  sensors  in  correctly  classifying 
anomalies  as  either  TOI  or  non-TOI  using  the  cued  survey  data. 

•  Comparing  the  potential  ability  to  correctly  classify  anomalies  as  either  TOI  or  non-TOI 
using  the  dynamic  advanced  EMI  sensor  data. 

•  Selecting  a  dig/no-dig  threshold  that  recognizes  all  TOI  (no  false  negatives/Type  II  error) 
while  minimizing  the  number  of  false  alarms  {i.e.,  minimizing  false  positives/Type  I 
errors), 

•  Minimizing  the  number  of  targets  classified  as  “can’t  analyze”,  and 

•  Correctly  estimating  target  parameters  such  as  polarizabilities,  location,  depth,  and  size  of 
the  identified  anomalies. 

ARC  ADIS  performed  the  following  tasks  in  order  to  achieve  this  overall  objective: 

•  Dynamic  DGM  data  using  the  Geonics  EM61-MK2,  MetalMapper,  and  the  TEMTADS; 

•  Static,  cued  target  interrogation  using  both  the  MetalMapper  and  TEMTADS; 

•  Processing  and  quality  control  (QC)  of  dynamic  and  static  geophysical  data; 

•  Target  reacquisition; 

•  Intrusive  investigation  of  all  targets; 

•  Demolition  of  identified  MEC;  and 

•  Advanced  classification  of  the  static  advanced  EMI  data. 

1.3  REGULATORY  DRIVERS 

The  Military  Munitions  Response  Program  (MMRP)  is  charged  with  characterizing  and,  where 
necessary,  remediating  MRSs.  When  an  MRS  is  remediated,  it  is  typically  mapped  with  a 
geophysical  system,  {i.e.,  either  a  magnetometer  or  EMI  sensor),  and  the  locations  of  all 
detectable  signals  are  excavated.  Many  of  these  detections  do  not  correspond  to  munitions,  but 
rather  to  other  harmless  metallic  objects  or  geology:  field  experience  indicates  that  often  in 
excess  of  90%  of  objects  excavated  during  the  course  of  a  MR  are  found  to  be  nonhazardous 
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items.  Current  geophysical  technology,  as  it  is  traditionally  implemented,  does  not  provide  a 
physics-based,  quantitative,  validated  means  to  discriminate  between  hazardous  munitions  and 
nonhazardous  items. 

With  no  information  to  suggest  the  origin  of  the  signals,  all  anomalies  are  currently  treated  as 
though  they  are  intact  munitions  when  they  are  dug.  They  are  carefully  excavated  by  certified 
UXO  technicians  using  a  process  that  often  requires  expensive  safety  measures,  such  as  barriers 
or  exclusion  zones.  As  a  result,  most  of  the  costs  to  remediate  a  munitions-impacted  site  are 
currently  spent  on  excavating  targets  that  pose  no  threat.  If  these  items  could  be  determined  with 
high  confidence  to  be  nonhazardous,  some  of  these  expensive  measures  could  be  eliminated  or 
the  items  could  be  left  unexcavated  entirely. 

The  MMRP  is  severely  constrained  by  available  resources.  Remediation  of  the  entire  inventory 
using  current  practices  is  cost  prohibitive,  within  current  and  anticipated  funding  levels.  With 
current  planning,  estimated  MR  completion  dates  on  many  sites  are  decades  out.  The  Defense 
Science  Board  observed  in  its  2003  report  that  significant  cost  savings  could  be  realized  if 
successful  classification  between  munitions  and  other  sources  of  anomalies  could  be 
implemented  (OSDA,  2003).  If  these  savings  were  realized,  the  limited  resources  of  the  MMRP 
could  be  used  to  accelerate  the  remediation  of  MRSs  that  are  currently  forecast  to  be  untouched 
for  decades. 
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2.0  TECHNOLOGY 


This  demonstration  consisted  of  dynamic  and  static  data  collection  with  a  total  of  three 
geophysical  sensor  systems  and  analysis  using  conventional  data  processing  methods  to  identify 
geophysical  anomalies  and  advanced  data  processing  to  extract  features  and  perform  anomaly 
classification.  Details  of  each  technology  and  a  brief  description  of  the  major  components  of  the 
demonstration  are  provided  below. 

2.1  TECHNOLOGY  DESCRIPTION 
2.1.1  Geonics  EM61-MK2 

The  EM61-MK2  is  a  time-domain  EMI  sensor  that  transmits  a  current  through  an  electrical  loop 
that  induces  a  primary  magnetic  field  that  magnetizes  buried  (or  surface)  objects.  Turning  off 
the  transmit  current  causes  an  abrupt  change  in  the  magnetic  field  that  excites  eddy  currents 
within  the  metallic  object.  These  eddy  currents  decay  as  a  function  of  time  and  are  recorded  by 
four  receiver  time  gates. 

The  Geonics  EM61-MK2  is  the  industry  standard  geophysical  instrument  for  collecting  DGM 
data  at  MRSs.  It  consists  of  a  lower  0.5-meter  (m)  by  1.0-m  transmit/receive  coil  and  an  upper 
0.5-m  by  1.0-m  receive  coil  (see  Figure  2-1).  ARCADIS  collected  dynamic  DGM  data  across 
the  entire  5 -acre  demonstration  site  using  the  off-the-shelf  EM61-MK2  technology. 


Figure  2-1:  Geonics  EM61-MK2 
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2,1,2  Geometries  MetalMapper 

The  Geometries  MetalMapper  is  the  first  commercially  available  advanced  EMI  sensor  designed 
specifically  for  the  purpose  of  advanced  classification.  It  consists  of  three  orthogonal  1-m^ 
transmit  coils  and  seven  10  centimeter  (cm),  3-component,  orthogonal  receiver  coils  (Figure  2- 
2).  The  system  was  developed  in  collaboration  with  ESTCP  (Prouty,  2011)  and  was  validated 
during  the  ESTCP  live  demonstration  at  the  former  Camp  San  Luis  Obispo  (Nelson  et.  al,  2010; 
Prouty,  2009)  and  other  live-sites  to  be  effective  at  correctly  classifying  TOI  and  non-TOI. 
ARCADIS  operated  the  MetalMapper  in  dynamic  detection  mode  across  the  entire  site  and  static 
classification  mode  for  approximately  one  half  of  the  detected  targets  during  the  CEMR  live-site 
demonstration.  ARCADIS  used  the  commercially  available  MetalMapper  without  making 
modifications  to  the  system. 


Figure  2-2:  Geometries  MetalMapper 
2,1,3  NRL’s  TEMTADS  MP  2x2 

The  TEMTADS  is  a  MP  advanced  EMI  sensor  array  (see  Figure  2-3)  based  on  NRL’s  larger, 
5x5  TEMTADS  array.  The  TEMTADS  MP  2x2  consists  of  four  35cm  transmit  coils  with  four  8 
cm  tri-axial  receiver  cubes.  The  TEMTADS  MP  2x2  was  developed  through  ESTCP  (Kingdon, 
2012)  and  has  been  shown  to  reliably  retain  the  performance  of  the  original  TEMTADS  in  a 
much  smaller  size,  which  enables  the  MP  version  to  access  difficult  terrain  where  mobility  is 
limited  (ESTCP,  2012a  and  2012b;  Kingdon,  2012).  ARCADIS  operated  the  TEMTADS  system 
in  dynamic  detection  mode  across  approximately  1  acre  in  the  target  area  and  static  classification 
mode  for  approximately  one  half  of  the  detected  targets  during  the  CEMR  live-site 
demonstration.  ARCADIS  used  the  TEMTADS  without  making  modifications  to  the  system. 
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Figure  2-3:  NRL’s  TEMTADS  MP  2x2 

2.2  TECHNOLOGY  DEVELOPMENT 

ARCADIS  performed  a  live  site  demonstration  at  CEMR  using  existing  technologies  and  did  not 
develop  new  instrument  technologies  during  this  demonstration.  Technologies  used  during  this 
demonstration  include  the  above  three  geophysical  sensors,  real  time  kinematic  (RTK) 
differential  global  positioning  system  (DGPS),  and  the  UX-Analyze  Advanced  module  within 
Geosoft  Oasis  Montaj®.  The  MetalMapper,  TEMTADS,  and  UX-Analyze  Advanced  were 
developed  under  ESTCP  and  further  descriptions  of  their  development  can  be  found  in  the 
following  reports; 

•  MetalMapper;  Prouty,  2011; 

•  TEMTADS;  Kingdon  et.  al,  2012; 

•  UX-Analyze  Advanced;  Keiswetter,  2009,  and  the  following  projects,  which  do  not 
currently  have  technical  reports  that  are  available; 

o  MR-201164;  Demonstration  of  Physics-Inspired  Classification  Methodologies  for 
MR 

o  MR-201312;  UXO  Classification  Demonstrations  at  Live  Sites  Using  UX- 
Analyze 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  major  advantage  of  the  advanced  EMI  sensors  and  UX-Analyze  Advanced  module  is  that 
combined,  they  provide  the  ability  to  classify  anomalies  as  being  due  to  either  TOI  or  non-TOI. 
This  can  lead  to  significant  cost  savings  in  MR  cleanups.  Conventional  DGM  sensors  (e.g., 
EM61-MK2)  have  very  limited  ability  to  correctly  classify  TOI  and  non-TOI.  Other  advanced 
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EMI  sensors  (e.g.,  Berkeley  UXO  Diseriminator)  have  also  been  successful  in  SERDP  and 
ESTCP  funded  classification  demonstrations;  however,  they  were  not  used  during  this  live  site 
demonstration  due  to  limited  availability.  Part  of  this  investigation  was  to  better  determine  the 
advantages  of  each  of  the  EMI  sensors  operation.  Table  2-1  shows  the  advantages  and 
limitations  of  each  of  the  advanced  EMI  sensor  technologies  deployed  at  CEMR.  Discussions  of 
quantitative  production  rates  for  each  sensor  are  presented  in  Section  6.0. 
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Table  2-1:  Qualitative  Advantages  and  Disadvantages  of  Deployed  Technology 


Category 


General 


Dynamic 

Surveys 


Static,  Cued 
Interrogation 


Advantages 

and 

Limitations 


Portability 


Sensor 
readiness  for 
field 

deployment 


Dynamic 

Survey 

Efficiency 


EM61-MK2 


Can  be  implemented  in  MP 
mode. 


Sturdy  instrument  with 
considerable  use  throughout  the 
industry. 


Quickest  and  has  highest  user 
familiarity  of  the  dynamic 
survey  modes.  Requires  setup 
of  ropes  for  straight  line 
positioning. 


Dynamic 

Sensor 

Deployment 


Dynamic 

Advanced 

Classification 

Potential 


Static 

Advanced 

Classification 

Potential 


Static  Sensor 
Deployment 


MetalMapper 


Limited  set  up  time,  has  real  time 
speed  monitoring,  and  doesn't 
require  ropes  for  straight-line 
profiling. 


Use  of  the  skid  ensures  sensor  is 
close  to  the  ground  surface  and 
maintains  a  relatively  constant 
height  above  the  ground. 


Yes 


TEMTADS  MP  2x2 


Can  be  implemented  in  MP  mode. 


Field  computer  is  rugged;  however, 
backpack  is  relatively  heavy.  Field 
components  are  not  currently  easy  to 
replace.  Prototype  design  was  not  fit  for 
rough  terrain  or  potentially  cold  weather 
where  plastic  parts  might  break. 


Requires  the  use  of  ropes  for  straight  line 
positioning. 


Yes 


Sensor  can  be  placed  within  1 5 
cm  of  the  ground  surface  using 
the  skid. 


Yes 


Yes 


Sensor  height  placed  the  sensors  within 
20  cm  of  the  ground  surface. 
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Category 

Advantages 

and 

Limitations 

EM61-MK2 

MetalMapper 

TEMTADS  MP  2x2 

Reacquisition 

procedures 

Integrated 

Notes: 

•  Green  highlighted  cells  indicate  a  relative  advantage  of  the  sensor. 

•  Yellow  highlighted  cells  indicate  a  relatively  neutral  situation  (z.e.,  neither  an  advantage  nor  a  limitation). 

•  Red  highlighted  cells  indicate  a  relative  limitation  of  the  sensor. 

1  -  The  TEMTADS  has  since  been  upgraded  with  wheels  on  the  front  of  the  sensor  that  likely  corrects  this  limitation 
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3.0  PERFORMANCE  OBJECTIVES 


The  performance  objectives  for  this  demonstration  are  summarized  in  Table  3-1.  There  are 
objectives  for  the  data  collection  and  data  analysis  portions  of  the  demonstration. 

In  the  Analysis  and  Classification  Objectives  section,  the  first  three  objectives  refer  to  the 
classification  part  of  the  demonstration  with  the  first  two  referring  to  the  best  results  from  each 
approach  in  a  retrospective  analysis  and  the  third  addressing  how  well  each  demonstrator  is  able 
to  specify  the  correct  threshold  in  advance.  The  final  two  objectives  refer  to  the  feature 
extraction  part  of  the  demonstration. 

Table  3-1:  Performance  Objectives  for  This  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Data  Collection  Objectives 

Spatial  coverage  in 
detection  survey 

Extended  footprint 
coverage 

•  Mapped  survey 
data 

98%  coverage 

Along-line 

measurement  spacing 

Point-to-point  spacing 
from  data  set 

•  Mapped  survey 
data 

98%  of  data  <  0.15  m 

Detection  of  all  TOI 

Percent  detected  of 
seeded  items 

•  Eocation  of  seeded 
items 

•  Anomaly  list 

100%  of  seeded  items 
detected  within  0.6  m 
halo 

Repeatability  of 
instrument 
verification  strip 
(IVS)  measurements 

Amplitude  of  EM 
anomaly 

Amplitude  of 
polarizabilities 

•  Twice-daily  IVS 
survey  data 

Detection:  Amplitude 
within  25% 
Down-track  location 
±25  cm 

Cued:  Polarizabilities 
±10% 

Production  rate 

Number  of  acres  of 
data  collection  per 
day 

Time  required  to 
analyze  each  target 

•  Eog  of  field  work 
and  data  analysis 
time  accurate  to  15 
minutes 

Survey:  2  acres  per 
day 

Analysis  time:  <5 
minutes  per  target 
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Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Cued  interrogation  of 
anomalies 

Instrument  position 

•  Cued  survey  data 

100%  of  anomalies 
where  the  center  of 
the  instrument  is 
positioned  within  40 
cm  of  actual  target 
location 

Analysis  and  Classification  Objectives 

Maximize  correet 
elassification  of  TOl 

Number  of  TOl 
retained 

•  Ranked  anomaly 
lists 

•  Scoring  reports 
from  Institute  for 
Defense  Analyses 
(IDA) 

Approach  correctly 
classifies  all  TOl 

Maximize  correet 
classification  of  non- 
TOI 

Number  of  false 
alarms  eliminated 

•  Ranked  anomaly 
lists 

•  Scoring  reports 
from  IDA 

Reduction  of  clutter 
digs  required  by 
>75%  while  retaining 
all  TOl 

Specification  of  no¬ 
dig  threshold 

Probability  of  correct 
classification  of  TOl 
and  number  of  false 
alarms  at 
demonstrator 
operating  point 

•  Demonstrator  - 
specified  threshold 

•  Scoring  reports 
from  IDA 

Threshold  specified 
by  the  demonstrator  to 
achieve  criteria  above 

Minimize  number  of 
anomalies  that  cannot 
be  analyzed 

Number  of  anomalies 
that  must  be  classified 
as  “Cannot  Analyze” 

•  Demonstrator  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >  95% 
of  anomalies  on  the 
detection  list. 

Correct  estimation  of 
target  parameters 

Accuracy  of  estimated 
target  locations  for 
seed  items 

•  Target  parameters 

•  Results  of  intrusive 
investigation 

X,  Y  <15  cm  (1g) 

Z  <10  cm  (1g) 
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3.1  OBJECTIVE:  SPATIAL  COVERAGE  FOR  DETECTION 

Detection  surveys  will  cover  the  entire  area  of  interest  so  that  all  detectable  targets  are  detected. 
Targets  are  detectable  if  the  transmitted  field  is  sufficiently  strong  to  reach  the  target  and  if  the 
measured  target  response  is  sufficiently  strong  in  return  to  exceed  a  given  threshold. 

3.1.1  Metric 

The  footprint  of  the  detection  survey  systems  will  be  compared  with  the  surface  area  for  the 
region  to  be  studied  in  survey  mode. 

3.1.2  Data  Requirements 

The  geographic  coordinates  for  the  perimeter  of  the  region  to  be  surveyed  and  the  survey  track 
will  be  utilized. 

3.1.3  Success  Criteria 

Success  completion  of  this  objective  requires  98%  spatial  coverage. 

3.2  OBJECTIVE:  ALONG-LINE  MEASUREMENT  SPACING 

The  reliability  of  the  survey  data  depends  on  the  density  of  coverage  of  the  site.  This  objective 
concerns  the  ability  of  the  instrument  operator  to  collect  data  with  acceptable  along-line 
measurement  spacing. 

3.2.1  Metric 

The  metrics  for  this  objective  are  the  percentage  of  data  points  within  an  acceptable  along-line 
spacing. 

3.2.2  Data  Requirements 

A  mapped  data  fide  will  be  used  to  judge  the  success  of  this  objective. 

3.2.3  Success  Criteria 

Successful  completion  of  this  objective  requires  98%  of  the  data  to  have  an  along-line 
measurement  spacing  of  less  than  0.15  m. 

3.3  OBJECTIVE:  DETECTION  OF  ALL  TOI 

Quality  data  should  lead  to  a  high  probability  of  detecting  all  TOI  at  the  site. 

3.3.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the  specified 
anomaly  selection  threshold. 

3.3.2  Data  Requirements 

ARCADIS  will  prepare  an  anomaly  list.  IDA  personnel  will  score  the  detection  probability  of 
the  seeded  items. 
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3,3,3  Success  Criteria 

Successful  completion  requires  that  100%  of  the  seeded  items  are  deteeted  within  a  halo  of  0.6 

m. 


3.4  OBJECTIVE:  REPEATABILITY  OF  IVS  MEASUREMENTS 

The  reliability  of  the  data  also  depends  on  the  proper  funetioning  of  the  equipment.  This 
objeetive  eoneems  the  twiee-daily  confirmation  of  sensor  system  performanee. 

3.4.1  Metric 

The  metries  for  this  objeetive  are  the  amplitude  and  down-traek  position  of  the  maxima  for  the 
advaneed  systems  in  survey  mode  and  the  standard  deviation  of  the  polarizabilities  for  the 
advaneed  systems  in  cued  mode  obtained  from  eaeh  of  the  twiee-daily  surveys  of  the  IVS. 

3.4.2  Data  Requirements 

The  data  will  be  used  to  judge  this  objeetive. 

3.4.3  Success  Criteria 

The  objeetive  will  be  eonsidered  met  for  the  EM61-MK2  and  advanced  systems  in  survey  mode 
if  the  measured  amplitudes  for  eaeh  objeet  are  within  25%  of  the  mean  and  the  down-traek 
position  of  the  anomaly  peak  is  within  25  em  of  the  known  loeation.  The  objeetive  will  be 
eonsidered  met  for  the  advaneed  systems  in  eued  mode  if  the  standard  deviation  of  the  estimated 
polarizabilities  is  within  10%  of  the  mean. 

3.5  OBJECTIVE:  PRODUCTION  RATE 

3.5.1  Metric 

The  metric  for  this  objective  is  the  number  of  aeres  of  data  colleetion  per  day  and  the  time 
required  to  analyze  eaeh  target  of  cued  survey  data. 

3.5.2  Data  Requirements 

ARCADIS  will  provide  the  Program  Offiee  a  daily  log  of  field  work  and  data  analysis  with  time 
aeeurate  to  15  minutes  to  detail  what  work  was  performed  on  site.  The  Program  Offiee  will 
review  the  daily  logs  and  provide  feedbaek  if  time  reeorded  on  the  logs  is  not  aeeurate  enough. 

3.5.3  Success  Criteria 

The  objective  will  be  eonsidered  to  be  met  if  the  produetion  rate  of  1  aere/day  is  met  during 
dynamie  data  aequisition  and  if  the  production  rate  is  less  than  5  minutes  per  target  on  average 
for  the  eued  anomaly  interrogation. 

3.6  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

The  reliability  of  cued  survey  data  depends  on  aeeeptable  instrument  positioning  during  data 
eolleetion  in  relation  to  the  aetual  anomaly  loeation. 
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3.6,1  Metric 


The  metric  for  this  objective  is  the  percentage  of  anomalies  that  are  within  the  acceptable 
distance  of  the  center  of  the  instrument  during  data  collection  from  the  actual  target  location. 

3.6.2  Data  Requirements 

ARCADIS  will  provide  the  Program  Office  a  weekly  list  of  the  location  of  the  center  of  the 
instrument  for  each  cued  anomaly  interrogated  in  the  preceding  week.  The  Program  Office  will 
review  the  offsets  for  the  QC  seeds  and  provide  feedback  if  the  cued  sensor  was  not  within  the 
acceptable  distance.  ARCADIS  will  reacquire  data  for  those  anomalies  and  perform  a  root  cause 
analysis  for  each  failure. 

3.6.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  center  of  the  instrument  is  positioned  within  40 
cm  of  the  actual  anomaly  location  for  100%  of  the  cued  anomalies. 

3.7  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach.  By 
collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  targets  will  be  classified  with  high  efficiency.  This  objective  concerns 
the  component  of  the  classification  problem  that  involves  correct  classification  of  TOI. 

3.7.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  anomaly  list  for  a  particular  sensor 
that  can  be  correctly  classified  as  TOI  by  each  classification  approach. 

3.7.2  Data  Requirements 

ARCADIS  will  prepare  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list.  IDA 
personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.7.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  all  of  the  TOI  are  correctly  labeled  as  TOI  on  the 
ranked  anomaly  list. 

3.8  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms,  targets  will  be  classified  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  false  alarm 
reduction. 
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3.8.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  sensor  dig  list  that  can  be  correctly 
classified  as  non-TOI  by  each  classification  approach. 

3.8.2  Data  Requirements 

ARCADIS  will  prepare  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list.  IDA 
personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.8.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  75%  of  the  non-TOI  items  can  be 
correctly  labeled  as  non-TOI  while  retaining  all  of  the  TOI  on  the  dig  list. 

3.9  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospective  analysis  as  will  be  performed  in  this  demonstration,  it  is  possible  to  tell  the  true 
classification  capabilities  of  a  classification  procedure  based  solely  on  the  ranked  anomaly  list 
submitted  by  ARCADIS.  In  a  real-world  scenario,  all  targets  may  not  be  dug  so  the  success  of 
the  approach  will  depend  on  the  ability  of  an  analyst  to  accurately  specify  their  dig/no-dig 
threshold. 

3.9.1  Metric 

The  probability  of  correct  classification  of  TOI,  Probability  of  Classification  (Pdass),  and  number 
of  false  alarms.  Number  of  False  Alarms  (Nfa),  at  the  ARCADIS-specified  threshold  are  the 
metrics  for  this  objective. 

3.9.2  Data  Requirements 

ARCADIS  will  prepare  a  ranked  anomaly  list  with  a  dig/no-dig  threshold  indicated.  IDA 
personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.9.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  75%  of  the  non-TOI  items  can  be 
correctly  labeled  as  non-TOI  while  retaining  all  of  the  TOI  at  the  ARCADIS-specified  threshold. 

3.10  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce  the  effectiveness  of 
the  classification  process. 

3,10,1  Metric 

The  number  of  anomalies  for  which  reliable  parameters  cannot  be  estimated  is  the  metric  for  this 
objective. 
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3.10.2  Data  Requirements 

ARCADIS  will  provide  a  list  of  all  parameters  as  part  of  their  results  submission  along  with  a  list 
of  those  anomalies  for  whieh  parameters  eould  not  be  reliably  estimated. 

3.10.3  Success  Criteria 

The  objeetive  will  be  eonsidered  to  be  met  if  reliable  parameters  ean  be  estimated  for  >  95%  of 
the  anomalies  on  the  sensor  anomaly  list. 

3.11  OBJECTIVE:  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  target  parameters  that  are  estimated  in  the  first  phase 
of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately. 

3.11.1  Metric 

Accuracy  of  estimation  of  target  parameters  is  the  metric  for  this  objective. 

3.11.2  Data  Requirements 

ARCADIS  will  compare  their  estimated  parameters  for  the  seed  items  to  those  expected. 

3.11.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  estimated  X,  Y  locations  are  within  15  cm  (la) 
and  the  estimated  depths  are  within  10  cm  (la). 
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4.0  SITE  DESCRIPTION 

The  site  deseription  material  reprodueed  here  is  taken  from  the  Final  Limited  Site  Investigation 
Report,  Former  Camp  Ellis  Military  Range,  Table  Grove,  Illinois  (CH2MHill,  2009).  More 
details  ean  be  obtained  in  that  report.  The  former  CEMR  is  loeated  in  Fulton  County,  Illinois, 
between  the  towns  of  Ipava  and  Table  Grove  in  western  Illinois.  The  eamp  area  covers 
approximately  17,455  acres,  and  the  terrain  of  the  former  facility  varies.  Most  of  the  former 
CEMR’s  land  area  is  used  for  farming,  though  some  parts  of  the  former  site  also  are  used  as 
pastureland,  and  a  few  wooded  areas  exist  in  the  northern  portion.  Most  of  the  area  is 
undeveloped,  occupied  by  corn  and  bean  fields.  Some  parts  of  the  former  cantonment  area 
remain  unfarmed  because  the  thick  concrete  slabs  used  as  building  foundations  are  difficult  to 
remove.  Tree  groves  exist  in  other  areas. 

A  subset  of  the  Rocket,  Rifle,  and  Hand  Grenades  MRS  (Area  A)  was  selected  for  the  ESTCP 
study.  Area  A  is  located  in  the  northwest  quadrant  of  the  former  CEMR  and  consists  of 
approximately  52  acres  of  primarily  cropland.  Figure  4-1  shows  the  former  CEMR  and  the 
location  of  Area  A. 

4.1  SITE  SELECTION 

The  location  of  the  Area  A  demonstration  site  is  shown  in  Figure  4-2,  while  Figure  4-3  shows 
the  Area  A  geophysical  anomaly  density,  which  was  calculated  from  a  geophysical  transect 
survey  conducted  as  part  of  the  ongoing  MR  Remedial  Investigation  at  the  former  CEMR.  The 
anomaly  density  data  shows  one  elevated  anomaly  density  area  relative  to  background.  This  area 
of  higher  anomaly  density  was  interpreted  to  be  a  potential  target  area  and  ARCADIS 
investigated  5  acres  within  and  around  the  potential  target  area.  The  proposed  boundary  of  the 
area  of  investigation  is  shown  on  Figure  4-3. 

4.2  SITE  HISTORY 

The  War  Department  began  surveying  Fulton  County  in  the  fall  of  1941  and  selected  the  specific 
site  for  the  camp  in  early  1942.  Construction  began  on  the  camp  in  September  1942  and 
continued  through  the  winter.  Most  of  the  area  to  be  appropriated  was  farmland,  although  it 
included  the  small  village  of  Bemadotte.  The  completed  camp,  originally  intended  to  be  a 
75,000-acre  army  training  facility,  encompassed  17,455  acres  and  had  more  than  2,300 
buildings,  1,100  of  which  were  coal-heated  barracks. 

Camp  Ellis  trained  a  wide  variety  of  soldiers  for  World  War  II.  The  camp  had  different  small 
arms  ranges,  including  four  1 ,000-inch  courses,  a  transition  range,  combat  ranges,  a  target  pistol 
range,  a  submachine-gun  course,  a  miniature  anti-aircraft  range,  two  infiltration  courses,  two 
bazooka  and  rifle  grenade  ranges,  two  live  hand  grenade  courts,  and  a  small  “German  Village”  to 
train  troops  in  the  detection  of  land  mines  and  booby  traps. 

Training  at  Camp  Ellis  reached  its  pinnacle  in  June  1944.  Upon  completion  of  their  training, 
units  were  dispatched  to  the  European  and  Pacific  theaters.  In  January  1945,  the  engineer  group 
stationed  at  the  CEMR  was  disbanded,  and  other  units  trained  at  the  camp  soon  followed.  The 
camp,  however,  remained  open  with  the  primary  mission  of  guarding  Prisoners  of  War. 
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Figure  4-1:  Aerial  photograph  of  the  demonstration  site.  A  subset  of  Area  A  was  selected  for  the  ESTCP  study. 
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Figure  4-2:  Area  A  Site  Location  Map 
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Figure  4-3:  ESTCP  Investigation  Area  Boundary  with  Geophysical  Anomaly  Density 
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Some  of  the  land  assoeiated  with  Camp  Ellis  was  allocated  to  local  farmers  as  early  as  1945,  and 
the  entire  camp  was  declared  surplus  by  October  1945.  In  April  1946,  the  War  Department 
opted  to  keep  Camp  Ellis  open  as  an  Army  Ground  Eorces  training  center.  In  1950,  the 
Department  of  Defense  (DoD)  screened  the  property  for  other  government  use,  concluding  that 
Camp  Ellis  should  be  considered  surplus;  the  buildings  were  sold  in  June  1950.  The  General 
Services  Administration  (GSA)  sold  2,000  acres  of  the  property  back  to  the  original  owners  in 
late  1954.  In  January  1955,  GSA  held  an  auction  to  sell  4,000  additional  acres,  and  the 
remaining  property  was  sold  throughout  that  year. 

4.3  SITE  GEOLOGY 

There  are  three  stratigraphic  units  at  the  former  CEMR:  shallow  soil,  glacial  till,  and  bedrock 
(CH2MHill,  2009).  The  youngest  of  these  units  is  the  shallow  soil,  followed  by  the  underlying 
glacial  till,  and  then  by  bedrock,  which  is  at  greater  depths.  There  are  three  types  of  shallow 
soils  at  the  Area  A  MRS: 

•  Ipava  Silt  Eoam,  0  to  2  percent  slopes 

•  Sable  Silty  clay  loam,  0-2  percent  slopes 

•  Greenbush  silt  loam,  2  to  5  percent  slopes  (Nelson  et.  al,  2009) 

All  three  of  these  soil  types  consist  primarily  of  silt,  with  some  clay.  The  underlying  glacial  till 
is  between  6  to  15  meters  in  thickness  and  is  composed  of  a  mixture  of  sand,  silt,  clay,  and 
gravel.  The  bedrock  in  the  area  is  composed  of  sedimentary  rocks,  which  are  primarly  shale, 
sandstone,  and  limestone. 

Several  small  streams  flow  in  the  vicinity  of  the  Area  A  MRS;  however,  none  of  them  cross  the 
live  site  demonstration  area. 

The  site-specific  soil,  geology,  and  hydrogeology  did  not  present  a  challenge  to  the  technologies 
demonstrated  during  this  demonstration. 

4.4  MUNITIONS  CONTAMINATION 

Physical  evidence  of  2.36-mch  practice  rockets  has  been  found  in  Area  A.  Rifle  and  hand 
grenades  are  also  suspected  as  this  range  is  historically  identified  as  the  Rocket,  Rifle,  and  Hand 
Grenades  MRS  (CH2MHill,  2009). 


21 


5.0  TEST  DESIGN 

The  objective  of  this  program  is  to  demonstrate  a  methodology  for  the  use  of  classification  in  the 
MR  process.  The  three  key  components  of  this  methodology  are  collection  of  high-quality 
geophysical  data  and  principled  selection  of  anomalous  regions  in  those  data,  analysis  of  the 
selected  anomalies  using  physics-based  models  to  extract  target  parameters  such  as  size,  shape, 
and  material  properties,  and  the  use  of  those  parameters  to  construct  a  ranked  anomaly  list.  Each 
of  these  components  were  handled  separately  in  this  program. 

ARCADIS  collected  dynamic  DGM  and  cued  survey  data  for  analysis  andprocessed  the 
individual  dynamic  and  cued  survey  data  sets  using  existing  routines  to  extract  target  parameters. 
These  parameters  were  passed  to  the  classification  routines  that,  after  training  on  a  limited 
amount  of  site-specific  ground  truth,  were  used  to  produce  ranked  anomaly  lists. 

Since  this  is  a  live  site  demonstration,  all  anomalies  on  the  master  dig  list  were  intrusively 
investigated.  The  underlying  target  was  uncovered,  photographed,  located  with  a  cm-level  GPS, 
and  removed.  ARCADIS  was  able  to  request  ground  truth  data  for  training  of  our  classifier. 

At  the  conclusion  of  training,  ARCADIS  submitted  ranked  anomaly  lists  for  both  the 
MetalMapper  and  TEMTADS  data  sets.  All  anomalies  were  categorized  and  placed  on  the  dig 
list  in  the  following  order: 

•  Category  -1:  anomalies  for  which  training  labels  had  been  requested.  These  anomalies 
were  placed  at  the  top  of  the  dig  list. 

•  Category  0:  Anomalies  for  which  ARCADIS  was  not  able  to  extract  reliable  parameters. 

•  Category  1:  Anomalies  that  had  a  high  likelihood  of  being  a  TOI  based  on  the  ARCADIS 
classification  method. 

•  Category  2:  Anomalies  for  which  ARCADIS  was  unsure  whether  the  anomalies  were 
TOI  or  non-TOI.  Category  2  anomalies,  as  well  as  those  listed  above,  were  placed  above 
the  ARCADIS  dig  threshold. 

•  Category  3:  Anomalies  that  had  a  high  likelihood  of  being  clutter  items  and/or  a  low 
likelihood  of  being  TOI.  These  anomalies  were  placed  below  the  dig  threshold. 

These  inputs  were  scored  by  IDA  with  emphasis  on  the  number  of  items  that  were  correctly 
labeled  nonhazardous  (i.e.,  non-TOI)  while  correctly  labeling  all  TOI. 

The  primary  objective  of  the  demonstration  was  to  assess  how  well  ARCADIS  was  able  to  order 
our  ranked  anomaly  lists  and  specify  the  threshold  separating  high  confidence  clutter  from  all 
other  items.  All  fieldwork  was  conducted  in  accordance  with  (lAW)  the  approved 
Demonstration  Plan,  which  included  a  Site  Safety  and  Health  Plan  and  the  Explosives  Siting 
Plan. 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

ARCADIS  performed  the  following  tasks  in  order  to  achieve  this  overall  objective: 

•  Site  Preparation  {e.g.,  civil  surveying  and  vegetation  removal) 
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o  Civil  Surveys  (e.g.  establishing  First-order  Navigation  Points  and  site  boundary 
surveys). 

o  Surface  Clearance 
o  Establishing  the  IVS 
o  Seeding  the  Site 

•  Data  Calibration  (see  Section  5.4) 

o  Twice  daily  IVS  surveys 
o  Static  and  dynamic  test  pit  data  collection 
o  Blind  seeding  program 

•  Data  Collection  (see  Section  5,5) 

o  Dynamic  DGM  data  collection  using  the  Geonics  EM61-MK2,  MetalMapper,  and 
the  TEMTADS 

o  Static,  cued  target  data  collection  using  both  the  MetalMapper  and  TEMTADS 

•  Data  Analysis 

o  Pre-Processing  and  QC  (See  Section  6,1) 
o  Target  selection  for  detection  (See  Section  6.2) 
o  Parameter  estimation  (See  Section  6,3) 
o  Classification  and  classifier  training  (See  Section  6,4) 
o  Classification  (See  Section  6,5) 

•  Data  Products  (See  Section  6,6) 

o  Intrusive  investigation  dig  list 
o  Intrusive  investigation  results 
o  Raw  and  Processed  dynamic  and  static  data. 

5.2  SITE  PREPARATION 

Several  activities  occurred  prior  to  data  collection  to  ensure  the  resulting  data  would  support  a 
successful  demonstration.  These  activities  included  a  survey  of  historical  records,  a  civil  survey 
to  establish  survey  control  monuments  for  use  as  a  base  station  for  RTK  DGPS  and  to  survey  the 
site  boundary;  a  surface  sweep  to  remove  any  MEC  or  metallic  debris  from  the  surface; 
construction  of  an  IVS,  and  emplacement  of  blind  seed  items  (BSIs)  within  the  area  of 
investigation.  The  following  sections  provide  greater  details  on  each  of  these  site  preparation 
activities. 

5.2.1  Survey  of  Historical  Records 

Historical  information  on  this  site  has  been  referenced  to  the  Final  Limited  Site  Investigation 
Report,  Former  Camp  Ellis  Military  Range,  Table  Grove,  Illinois  (CH2MHih,  2009).  This 
report  was  posted  on  the  ESTCP  ETP  server  and  can  be  used  for  reference.  No  additional 
historical  records  were  reviewed  prior  to  the  start  of  the  Eive  Site  Demonstration  at  the  former 
CEMR. 

5.2.2  Civil  Surveys 

It  is  important  that  ah  survey  data  collection  and  validation  activities  were  conducted  on  a 
common  coordinate  system.  The  ESTCP  Program  Office  established  two  survey  control 
monuments  at  the  project  site.  The  location  information  and  data  sheet  for  the  two  first-order 


23 


control  monuments  that  were  used  for  this  demonstration  are  ineluded  as  Appendix  B  (see 
separate  Addendum).  After  the  ESTCP  Program  Offiee  established  the  eontrol  monuments, 
ARCADIS  subeontracted  additional  civil  surveying  aetivities  to  the  Farnsworth  Group,  Ine.,  a 
professionally  lieensed  surveyor  in  the  state  of  Illinois,  to  survey: 

•  IVS  seed  item  loeations, 

•  BSI  loeations,  and 

•  Southwest  eomers  of  eaeh  30-m  x  30-m  grid  established  for  DGM  surveys. 

The  loeations  of  the  each  of  these  items  are  eontained  in  Appendix  B  (see  separate  Addendum) 
and  Figure  5-1  shows  the  loeations  of  the  IVS  seed  items  and  DGM  grid  boundaries  relative  to 
the  ESTCP  demonstration  area. 

5.2.3  Surface  Clearance  and  Vegetation  Removal 

Prior  to  collecting  DGM  data,  ARCADIS’  UXO  technicians  conducted  a  surface  clearance 
across  the  entire  5-acre  demonstration  area  (see  Figure  5-1)  to  remove  metallic  debris  from  the 
surface  prior  to  the  DGM  surveys.  The  surface  clearance  removed  all  metal  objects  that  were  on 
the  ground  surface  or  partially  buried  which  may  interfere  with  the  DGM  data. 

The  demonstration  area  was  open  farmland  and  it  was  anticipated  prior  to  mobilization  that 
vegetation  removal  would  not  be  required  during  the  demonstration.  Once  the  DGM  survey 
crew  mobilized  to  the  site;  however,  the  DGM  team  determined  that  com  stalks  on  the  site  were 
interfering  with  DGM  data  collection.  In  particular,  the  combined  height  of  the  com  rows  and 
com  stalks  were  high  enough  that  the  com  stalks  often  hit  the  bottom  receiver  coil  of  the  EM61- 
MK2.  Therefore,  ARCADIS  paid  the  property  owner  to  mow  the  corn  stalks  with  a  bush  hog  to 
a  height  suitable  for  DGM  surveys.  After  the  corn  stalks  were  cut  down  close  to  the  ground 
surface,  ARCADIS  recollected  the  EM61-MK2  DGM  survey  data. 

5.2.4  IVS  Establishment 

ARCADIS  established  the  IVS  in  an  open  area  near  the  MRS  that  was  relatively  free  of 
background  anomalies.  Figure  5-1  shows  the  location  of  the  IVS  relative  to  the  MRS  boundary. 
The  IVS  was  established  lAW  the  Final  GSV  Report  (Nelson  et  ah,  2009). 

Prior  to  establishing  the  IVS,  the  field  team  used  the  Geonics  EM61-MK2,  operating  in  the 
Monitor/Null  mode,  to  determine  whether  the  proposed  IVS  lines  for  seed  items  and  a  line  for 
dynamic  background  noise  measurements  were  relatively  free  of  anomalies. 

The  IVS  consisted  of  one  hand  grenade,  two  small  industry  standard  objects  (ISOs),  one  partial 
steel  rifle  grenade  and  an  empty  hole  that  was  used  as  a  background  test  location  on  the  days  for 
which  static  data  was  collected  with  the  MetalMapper  and/or  TEMTADS.  The  ends  of  the  line 
were  marked  in  the  field  with  PVC  pin  flags.  The  seed  items  were  distributed  sufficiently  apart 
(approximately  10  feet)  to  prevent  overlapping  signals.  Table  5-1  presents  the  IVS  seed  item 
information,  including  the  seed  item  number,  description,  northing  and  easting  coordinates, 
depth  to  the  center  of  mass,  inclination  and  azimuth  of  each  seed  item.  Figure  5-2  shows  the 
IVS  layout.  Each  hole  was  dug  following  standard  anomaly  avoidance  procedures.  The  seed 
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Figure  5-1:  ESTCP  Investigation  Area  and  IVS  Location 
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items  were  placed  in  designated  orientation,  with  the  depth  measured  from  ground  surface  to  the 
item’s  center. 


Table  5-1:  Details  of  the  IVS 


Item 

ID 

Description 

Easting 

(m)i 

Northing 

(m)i 

Depth 

(cm) 

Inclination 

Azimuth 

(“clockwise 
from  North) 

T-OOl 

Hand  grenade 

720325.894 

4475481.962 

15 

Horizontal 

Across  Track 

T-002 

Small  ISO 

720320.82 

4475481.753 

10 

Horizontal 

Across  Track 

T-003 

Partial  steel 
rifle  grenade 

720315.903 

4475481.582 

15 

Horizontal 

Across  Track 

T-004 

Blank  Space 

720310.866 

4475481.368 

NA 

NA 

NA 

T-005 

Small  ISO 

720305.91 

447548 1. 1 86 

15 

Horizontal 

Across  Track 

Note: 

1  -  The  coordinates  are  provided  in  UTM  Zone  15N  World  Geodetic  System  84,  with  units  of 
meters. 
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Figure  5-2:  IVS  Layout 

All  items  were  buried  horizontally  in  the  least  favorable  orientation.  Depths  were  measured  in 
cm  and  extend  from  the  ground  surface  to  the  center  of  mass  of  the  seed  item. 

The  final  horizontal  location  of  the  center  of  the  seed  items  were  measured  by  a  PLS  with  a  RTK 
DGPS  rover  receiver  mounted  to  a  range  pole.  Final  seed  item  depths  were  measured  to  the 
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nearest  1/8  ineh  from  ground  surfaee  to  the  eenter  of  eaeh  seed  item.  The  item  parameters  {i.e., 
the  surveyed  loeation,  size,  depth,  orientation)  were  reeorded  and  entered  into  the  database.  The 
field  team  marked  the  loeation  of  the  seed  items  and  start  and  end  points  of  eaeh  IVS  line  at  the 
ground  surfaee  with  PVC  pin  flags  or  wooden  stakes. 

In  addition  to  the  IVS,  a  test  pit  was  dug  near  the  IVS  to  support  eolleetion  of  dynamie  and  cued 
{i.e.,  static)  MetalMapper  and  TEMTADS  data  over  known  TOl  at  known  depths,  orientations, 
and  locations  to  be  used  for  algorithm  training,  as  needed.  This  test  pit  data,  along  with  the  IVS 
data  was  used  to  establish  dynamic  detection  thresholds  for  the  dynamic  MetalMapper  and 
TEMTADS  data. 

5.2.5  Seeding  the  Site 

At  a  live  site  such  as  this,  the  ratio  of  clutter  to  TOl  is  such  that  only  a  small  number  of  TOl  may 
be  found;  far  from  enough  to  determine  a  demonstrator’s  classification  performance  with 
acceptable  confidence  bounds.  To  avoid  this  problem,  ARDCADIS’  UXO  technicians  emplaced 
BSls  in  the  demonstration  area  under  the  direction  of  the  ESTOP  Program  Office. 

The  demonstration  area  was  seeded  with  approximately  100  inert  munitions  {i.e.,  2.36”  Rockets, 
hand  grenades,  rifle  grenades,  and  ISOs).  All  seeds  were  initially  blind  to  ARCADIS 
geophysicists  involved  in  data  collection,  processing,  analysis,  and  classification  to  allow  for 
accurate  evaluation  of  ARCADIS’  classification  performance.  The  ESTOP  Program  Office  used 
a  subset  of  the  blind  seeds  to  QC  the  data.  If  QC  seed  failures  occurred,  ARCADIS  was  required 
to  conduct  a  root-cause  analysis  and  potentially  had  to  collect  data.  Eor  this  site,  the  BSl 
locations  were  only  known  to  the  ESTOP  Program  Office,  the  surveyors,  and  the  UXO 
technicians  responsible  for  seeding  the  site.  The  locations  and  depth  of  these  targets  were 
unknown  to  the  ARCADIS  Project  Manager  and  geophysical  team. 

The  exact  {x,  y)  location,  depth  to  the  center  of  the  target,  and  orientation  were  recorded  for  each 
emplaced  BSl  and  are  shown  in  Appendix  B  (see  separate  Addendum).  The  ESTOP  Program 
Office  chose  the  locations  for  the  BSls,  as  well  as  the  preferred  depth  at  which  each  BSl  would 
be  buried. 


5.3  SYSTEM  SPECIFICATION 
5.3.1  EM61-MK2 

The  Geonics  EM61-MK2  EMI  sensor  was  used  to  collect  dynamic  DGM  data  across  the  entire 
demonstration  site,  as  well  as  for  post-dig  anomaly  resolution  to  ensure  that  the  source(s)  of 
anomalies  were  recovered  during  intrusive  investigation.  The  EM61-MK2  was  operated  using 
the  standard,  factory-programmed  four  time  gates  on  the  bottom  coil  at  216,  366,  660,  and  1266 
microseconds  (ps).  Data  was  collected  at  a  frequency  of  10  Hertz  (Hz)  and  positioned  using  a 
Trimble  RTK  DGPS  that  was  updated  at  a  frequency  of  1  Hz.  Ropes  and  measuring  tapes  were 
used  to  assist  with  straight  line  profiling.  The  EM61-MK2  was  operated  lAW  the  Geonics 
operations  manual  and  the  approved  Demonstration  Plan. 
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5.3.2  MetalMapper 

The  MetalMapper  has  two  data  aequisition  modes:  Single-Point-Mode  (z.e.,  statie,  eued)  and 
Continuous-Mode  (z.e.,  dynamie).  ARC  ADIS  eolleeted  dynamie  data  on  0.5-m  line  spaeing 
aeross  the  entire  5  aere  site;  dynamie  data  on  0.4-m  line  spaeing  over  approximately  0.7  aeres  of 
the  site  eentered  on  the  target  area;  and  statie  data  over  1,195  targets  in  the  southern  portion  of 
the  site.  Of  the  1,195  statie  targets,  299  targets  were  eolleeted  with  both  the  MetalMapper  and 
the  TEMTADS  in  the  high  anomaly  density  area  in  the  target  area. 

Both  dynamie  and  statie,  eued  MetalMapper  data  were  eolleeted  at  CEMR.  Table  5-2  lists  the 
data  aequisition  parameters  that  were  used  to  eollect  data  in  both  modes.  In  single-point-mode 
the  system  eolleets  a  data  point  and  then  terminates  aequisition.  The  data  are  stored  as  a  single 
data  point  in  the  output  data  file.  In  eontinuous-mode,  the  system  initiates  eolleetion  of  a  new 
data  point  eoncurrently  with  eompletion  of  the  previous  data  point  and  continues  until  the 
operator  intervenes.  All  of  the  data  points  are  stored  to  the  same  output  data  file.  Data  were  then 
exported  to  the  TEM2CSV  software  where  they  were  converted  from  the  digital  .TEM  file  to  a 
comma  spaced  delimited  text  file  (.csv  file)  that  could  be  imported  into  Geosoft  Oasis  Montaj®. 
The  primary  processing  difference  between  dynamic  and  static  is  that  static  data  is  background 
corrected  in  TEM2CSV  and  dynamic  data  is  not.  Instead,  dynamic  data  is  leveled  {i.e., 
background  and  instrument  drift  are  removed)  in  Montaj®.  Data  processing  of  the  dynamic  data 
also  included  summing  the  z  component  of  the  inner  five  receiver  coils  for  channels  6  through 
11.  All  MetalMapper  data  was  positioned  with  an  RTK  DGPS  that  and  with  an  inertial 
measurement  unit  (IMU)  that  updated  at  a  frequency  of  1  Hz. 

5.3.3  TEMTADS  MP  2x2 

ARCADIS  collected  TEMTADS  data  in  dynamic  mode  over  approximately  0.7  acres  roughly 
centered  on  the  target  area  and  static  TEMTADS  data  at  1,087  anomalies  in  the  northern  portion 
of  the  site.  Of  the  1,087  anomalies,  299  targets  were  collected  with  both  the  MetalMapper  and 
the  TEMTADS  in  the  high  anomaly  density  area  in  the  target  area.  SAIC  converted  the  dynamic 
TEMTADS  data  into  a  Geosoft  XYZ  format  that  ARCADIS  imported  into  Geosoft  Oasis 
Montaj®  for  processing  using  the  UX-Detect  module. 

Table  5-3  lists  the  data  acquisition  parameters  that  were  used  to  collect  data  in  both  dynamic  and 
static  modes.  In  static  mode,  the  system  collects  a  data  point  and  then  terminates  acquisition. 
The  data  are  stored  as  a  single  data  point  in  the  output  data  file.  In  dynamic  survey  mode,  the 
system  initiates  collection  of  a  new  data  point  concurrently  with  completion  of  the  previous  data 
point  and  continues  until  the  operator  intervenes.  All  of  the  data  points  are  stored  to  the  same 
output  data  file.  Software  to  convert  dynamic  TEMTADS  data  was  not  commercially  available 
at  the  time  of  data  collection;  therefore,  SAIC  converted  the  raw  .tern  and  .csv  files  and  imported 
them  to  a  Geosoft  Oasis  Montaj®  database.  The  static  data  were  directly  imported  into  Geosoft 
Oasis  Montaj®,  where  background  corrections  were  applied  and  the  data  was  analyzed  using  the 
UX-Analyze  Advanced  module. 
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Table  5-2:  MetalMapper  Data  Acquisition  Parameters 


Mode 

Tx 

Coils 

Window 

Width 

Hold 

Off 

Time 

(us) 

Block 
Period  (s) 

Number 

of 

Repeats 

Block 

Length 

Number 
of  Stacks 

Decay 

Time 

(us) 

No. 

Gates 

Repeat 

Dynamic 

Z 

Only 

20% 

50 

0.1 

9 

0.1 

1 

2498 

18 

Continuous 

Static 

ZYX 

10% 

50 

0.9 

27 

0.9 

10 

8333 

50 

Once 

Table  5-3:  TEMTADS  Data  Acquisition  Parameters 


Mode 

Acq  Mode 

Gate 

Width 

Block  Period 
(s) 

Hold 

Off 

Time 

(us) 

Number  of 
Stacks 

Stack 

Period 

(s) 

Number 

of 

Repeats 

Decay 

Time 

(us) 

Number  of 
Gates 

Dynamic 

Decimated 

5% 

0.033 

50 

1 

0.1 

3 

2500 

19 

Static 

Decimated 

5% 

0.90 

50 

18 

0.9 

9 

2500 

121 
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5.4  CALIBRATION  ACTIVITIES 

Five  types  of  ealibration  aetivities  were  performed  at  CEMR: 

1)  Static  background  and  static  spike  QC  testing, 

2)  IVS  measurements, 

3)  Cued  background  measurements, 

4)  Test  pit  measurements,  and 

5)  UXO  Technician  instrument  functionality  testing. 

Each  of  these  calibration  activities  is  described  in  greater  detail  in  the  following  sections. 

5.4.1  Static  Background  and  Static  Spike  Testing 

QC  Static  background  and  spike  tests  were  collected  with  the  EM61-MK2  on  a  twice  daily  basis 
{i.e.,  at  the  beginning  and  end  of  each  day)  on  days  of  data  collection  to  verify  that  the 
instrument  was  operating  properly.  The  static  tests  were  conducted  by  placing  a  small  ISO  in  a 
horizontal  location  on  a  wooden  jig  placed  on  top  of  the  bottom  coil  to  ensure  that  the  test  item 
was  located  in  the  same  relative  location  for  each  test  measurement. 

The  EM61-MK2  static  background  and  static  spike  tests  were  not  included  in  the  performance 
metrics  outlined  within  the  approved  demonstration  plan;  however,  ARC  ADIS  evaluated  the  data 
using  the  following  industry  standard  performance  metrics: 

•  Static  BG:  no  exhibited  spikes, 

•  Static  Spike:  recorded  responses  within  10%  of  the  expected  value. 

All  of  the  static  data  collected  during  this  live  site  demonstration  met  these  performance  metrics. 
The  largest  variation  from  the  mean  of  the  static  spike  data  was  approximately  3.7%.  The  QC 
test  data  was  included  in  the  geophysical  data  deliverable  provided  to  the  ESTCP  Program  Office 
at  the  completion  of  the  field  effort  and  is  included  in  Appendix  C  (see  separate  Addendum)  of 
this  report. 

Although  static  background  and  spike  tests  were  not  collected  with  the  advanced  EMI  sensors, 
they  were  tested  at  the  IVS  on  a  twice  daily  basis  to  ensure  they  were  functioning  properly. 

5.4.2  Daily  IVS  Tests 

The  IVS  was  established  by  the  UXO  team  as  outlined  in  Section  5,2,4  and  was  surveyed  by  the 
field  geophysical  team  before  and  after  dynamic  and/or  static  geophysical  surveys  {i.e.,  morning 
and  evening  for  each  data  collection  day).  The  IVS  data  was  collected  in  order  to  verify  that  the 
equipment  was  operating  properly  and  that  the  response  of  the  instrument  met  the  performance 
metrics  established  in  the  demonstration  plan.  The  IVS  seed  item  descriptions  are  provided  in 
Table  5-1.  The  IVS  geophysical  data  was  provided  to  the  ESTCP  Program  Office  at  the 
conclusion  of  the  field  activities  and  is  included  in  Appendix  C  of  this  report.  Section  7,0  of 
this  report  discusses  whether  the  performance  metrics  were  met. 
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5.4.3  Background  Measurements 

Daily  calibration  efforts  for  the  advanced  sensors  also  eonsisted  of  colleeting  static  background 
(no  anomaly)  data  sets  periodically  throughout  the  day  at  quiet  spots  to  determine  the  system 
background  level  for  subtraction.  An  initial  set  of  baekground  spots  was  seleeted  from  the 
EM61-MK2  DGM  survey  data  and  vetted  with  the  advanced  EMI  sensors  prior  to  continued  use. 
In  general,  the  background  response  for  a  cued  target  was  subtracted  by  using  the  nearest 
background  dataset  (in  time)  to  the  cued  survey  data;  however,  if  QC  identified  issues  with  the 
background  data  set  (e.g.,  the  response  deeay  curve  did  not  return  to  zero),  the  next  nearest  (in 
time)  baekground  data  set  was  used  for  background  subtraction.  The  baekground  datasets  were 
provided  to  the  ESTCP  Program  Office  at  the  eonclusion  of  field  activities  and  are  included  in 
Appendix  C  (see  separate  Addendum)  of  this  report. 

5.4.4  Test  Pit  Calibration  Measurements 

Test  pit  measurements  of  targets  of  interest  were  made  with  the  MetalMapper  and  TEMTADS 
sensors  in  dynamic  and  static  mode.  Table  5-4  summarizes  the  test  pit  items,  approximate 
depths,  and  orientations  of  the  test  pit  items  that  were  collected  early  in  the  data  collection 
process.  Both  static  and  dynamic  data  were  collected  to  enable  the  test  pit  data  to  be  added  to 
site-specifie  classification  libraries.  ARCADIS  did  not  perform  classifieation  of  dynamic  data; 
therefore  only  the  static  data  was  added  to  the  MetalMapper  and  TEMTADS  libraries  during  this 
demonstration.  Appendix  C  (see  separate  Addendum)  contains  the  test  pit  measurements  for  the 
MetalMapper  and  TEMTADS  in  both  dynamic  and  static  mode. 

Table  5-4:  Test  Pit  Calibration  Data  Measurement  Summary 


Test  Pit  Item 

Approximate 
Depths  (cm) 

Orientations 

Empty  pit  (static  -  5  min) 

NA 

NA 

Small  ISO 

5,  10 

Nose  Up  (0°  Inclination),  Horizontal  (90; 
along-  and  across-track),  45°  Inclination, 
Nose  Down  (180°  Inclination) 

Hand  Grenade 

5,20.3 

Steel  Rifle  Grenade 

5,  15,25.4 

Aluminum  Rifle  Grenade 

5,  15,25.4 

2.36  inch  rocket 

10,  45.72 

5,4,5  UXO  Technician  Instrument  Functionality  Testing 

UXO  technicians  tested  analog  geophysical  instruments  used  during  intrusive  operations  at  the 
IVS  at  the  beginning  of  each  day  to  ensure  proper  instrument  functionality.  Analog  geophysical 
instruments  were  also  checked  to  ensure  instrument  sensitivity  was  adequate  to  detect  the 
anticipated  TOT  hollowing  these  checks,  settings  {i.e.,  sensitivity)  for  each  analog  sensor  was 
recorded  in  the  field  logbook  and  any  equipment  that  was  found  unsuitable  was  immediately 
removed  from  service. 
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5.5  DATA  COLLECTION 
5,5.1  Dynamic  Data  Collection 

5.5.1. 1.  Scale 

ARC  ADIS  performed  the  following  amounts  of  dynamie  surveys: 

•  EM61-MK2:  Entire  5  aere  site. 

•  MetalMapper  on  0.5-m  line  spaeing:  Entire  5  aere  site. 

•  MetalMapper  on  0.4-m  line  spaeing:  0.7  aeres  in  the  partial  investigation  area  roughly 
eentered  on  the  target  area. 

•  TEMTADS  data  on  0.4-m  line  spacing:  0.7  acres  in  the  partial  investigation  area  roughly 
centered  on  the  target  area. 

5.5.1.2,  Sample  Density 

ARCADIS  collected  dynamic  data  with  the  following  across-track  line  spacing: 

•  EM61-MK2:  0.75 -meters 

•  MetalMapper:  0.5-m  (entire  site)  and  0.4-m  (in  the  partial  investigation  area) 

•  TEMTADS:  0.4-m  in  the  partial  investigation  area. 

The  across-track  line  spacing  for  the  MetalMapper  was  decreased  from  0.75-m  to  0.5-m,  because 
only  the  inner  five  coils,  which  have  a  swath  of  0.5-m,  were  planned  to  be  used  in  data 
processing  and  target  selection.  The  0.5-m  line  spacing  ensured  there  was  overlap  of  the 
transmitter  coil  between  adjacent  lines.  The  TEMTADS  line  spacing  was  set  to  0.4-m,  because 
that  is  half  of  the  instrument  footprint.  The  additional  MetalMapper  data  collected  on  0.4-m  line 
spacing  was  collected  to  enable  comparisons  with  the  MetalMapper  data  collected  on  0.5-m  line 
spacing  to  determine  the  potential  effects  of  varying  the  line  spacing,  as  well  as  to  allow  data 
comparisons  to  the  TEMTADS  data. 

Table  5-5  presents  a  summary  of  the  speeds  achieved  with  each  of  the  dynamic  EMI  sensors. 
The  demonstration  plan  included  a  goal  of  having  speeds  less  than  0.75  meters  per  second  (m/s); 
however,  this  goal  was  not  listed  as  one  of  the  performance  objectives.  The  main  reasons  to 
have  a  speed  metric  are  to  ensure  that  the  along  line  measurement  spacing  metric  is  met  and  to 
minimize  the  potential  for  increased  noise  that  can  be  caused  by  increased  speeds.  Industry 
standard  is  that  95%  of  data  meet  whatever  the  speed  metric  is.  As  seen  in  Table  5-5,  the 
MetalMapper  data  met  the  speed  metric;  however,  the  EM61-MK2  and  TEMTADS  data  did  not. 
The  MetalMapper  field  computer  shows  the  speed  of  travel  in  real  time,  so  the  equipment 
operator  can  monitor  speed  to  ensure  data  is  collected  within  the  project’s  performance 
objectives.  The  other  two  instruments  did  not  have  this  capability,  at  least  at  the  time  of  data 
collection.  It  should  be  noted  that  with  the  along  line  sampling  density  of  0.15  m  and  the  10  Hz 
sampling  rate,  one  could  travel  1.5  m/s  and  still  meet  the  along  line  measurement  spacing.  In 
fact,  96.6%  of  the  TEMTADS  has  a  speed  of  less  than  or  equal  to  1.5  m/s,  which  is  consistent 
with  the  along  line  measurement  spacing.  Although  the  planned  speeds  were  not  accomplished 
for  all  datasets,  increased  noise  was  not  exhibited  in  the  data  and  the  down  line  and  across  line 


32 


sample  density  was  suffieient  to  the  meet  the  projeet’s  overall  objective  of  identifying  targets  for 
cued  interrogation  and  for  comparisons  between  the  various  datasets. 

Table  5-5:  Dynamic  EMI  Speed  Results 


Performance  Objective 

EM61- 

MK2 

MetalMapper 
(0.5m  line 
spacing) 

MetalMapper 
(0.4-m  line 
spacing) 

TEMTADS 

Speed  Metric  Success 
Criteria 

Speed  less  than  0.75  m/s 

Speed  Above  Success 
Criteria  (%) 

23.29% 

0.47% 

0.13% 

64.4% 

Percent  Below  Success 
Criteria  (%) 

76.61% 

99.53% 

99.87% 

35.6% 

Mean  Speed  (m/s) 

0.63 

0.18 

0.16 

0.89 

Standard  Deviation  (m/s) 

0.18 

0.1 

0.07 

0.4 

5.5.1.3.  Quality  Checks 

In  addition  to  the  calibration  activities  described  above  in  Section  5.4,  dynamic  mode  quality 

checks  included  the  following: 

•  At  the  beginning  of  each  day,  the  field  team  allowed  the  EMI  sensors  to  warm  up  for  a 
minimum  of  15  minutes. 

•  Static  Background  and  spike  measurements  were  collected  for  the  EM61-MK2  only  to 
ensure  repeatability  of  response.  Static  repeatability  of  the  advanced  sensors  were 
conducted  at  the  IVS. 

•  Eor  all  sensors,  dynamic  repeatability  tests  were  completed  at  the  IVS  on  a  twice  daily 
basis.  As  discussed  above,  ARCADIS  performed  twice  daily  instrument  function  checks 
at  the  IVS  to  verify  the  dynamic  response  and  positioning  repeatability  of  the  complete 
geophysical  system. 

5.5.1.4,  Data  Summary 

Eor  each  dataset,  the  field  team  created  a  file  using  the  date  and  a  sequential  alphabetic  character. 

The  naming  convention  varied  for  the  different  files  types  as  follows: 

•  EM61 

o  Data:  MMDDXX,  where 

■  MMDD  =  Month  and  Day 

■  XX  =  grid  number 
o  IVS:  MMDDIVSX,  where 

■  MMDD  =  month,  day 

■  X  =  I  (AM)  or  2  (PM) 

•  MetalMapper 
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o  Data:  EllisDynXXXXX.tem,  where 

■  XXXXX  =  Sequential  number 

o  I  VS  Filename:  EllisDynIVSXXXXX.tem,  where 

■  XXXXX  =  Sequential  number 

•  TEMTADS 

o  Data  and  IVS:  YYYYMMDD_XXXXX,  where 

■  YYYYMMDD  =  Year,  Month,  and  day. 

■  XXXXX  =  Sequential  number 

The  production  data  was  collected  on  individual  lines,  except  for  the  TEMTADS  data,  which 
was  collected  along  several  continuous  lines  that  resulted  in  collected  data  while  turning  around 
at  the  end  of  each  line. 

EM61  data  were  recorded  into  binary  file  formats  with  a  .p61  extension.  These  formats  were 
converted  into  the  final  .xyz  format  in  Geosoft.  MetalMapper  and  TEMTADS  data  were  both 
collected  in  .tern  format.  The  MetalMapper  data  was  converted  to  a  .csv  file  using  the 
TEM2CSV  program  and  then  imported  into  a  Geosoft  database  (.gdb)  for  processing.  SAIC 
used  proprietary  software  to  convert  the  dynamic  TEMTADS  data  from  the  raw  .tern  and  import 
it  into  a  Geosoft  database  (.gdb)  for  processing.  Geophysical  data  is  contained  in  Appendix  C 
(see  separate  Addendum)  and  has  been  organized  into  raw  and  processed  data  folders.  Each  of 
the  datasets  is  contained  in  a  single  .gdb  file.  Additional  information  on  the  organization  of  the 
data  is  contained  in  Appendix  C  (see  separate  Addendum). 

5,5.2  Static,  Cued  Survey  Data  Collection 

5.5.2. 1,  Scale 

ARC  ADIS  collected  static  data  at  a  total  of  1,983  targets.  299  of  the  targets  were  collected  with 
both  instruments;  therefore,  the  total  number  of  cued  targets  on  the  MetalMapper  and 
TEMTADS  target  lists  is  2,282.  ARCADIS  collected  the  following  amount  of  static  data: 

•  MetalMapper:  1,196  targets 

•  TEMTADS:  1,086  targets 

5.5.2.2,  Sample  Density 

Static  data  was  collected  at  an  individual  point  with  fixed  transmitter  and  receiver  geometry; 
therefore,  there  is  no  corresponding  along  line  or  across  line  data  density. 

5.5.2.3,  Quality  Checks 

In  addition  to  the  data  calibration  activities  described  in  Section  5,4,  ARCADIS  also  performed  a 
QC  of  the  static,  cued  survey  data  within  24  hours  of  data  collection  to  ensure  that  data  collected 
was  sufficient  for  advanced  classification  purposes.  As  part  of  this  QC,  the  data  processor 
verified  the  following: 

•  Background  data  location  did  not  exhibit  signs  of  a  local  piece  of  metal  within  the 
readings; 
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•  Static  rVS  decay  responses  were  within  performance  metrics; 

•  Static  IVS  classification  was  consistent;  and 

•  For  cued  targets,  verification  that  the  inverted  target  location  was  within  40  cm  of  the 
eenter  of  the  array. 

If  the  data  proeessor  found  indication  that  the  performance  metries  were  not  going  to  be  met 
{e.g.,  inverted  target  location  outside  instrument  footprint),  the  target  was  flagged  to  be  re- 
eollected  and  the  field  team  was  instructed  to  recollect  the  data. 

5,5.2,4,  Data  Summary 

For  each  dataset,  the  field  team  ereated  a  file  using  the  date  and  a  sequential  alphabetie  character. 
The  naming  convention  varied  for  the  different  files  types  as  follows: 

•  MetalMapper 

o  Background:  EllisBkgdXXXXX.tem,  where 

■  XXXXX  =sequential  number 
o  Data:  EllisStatXXXXX.tem,  where 

■  XXXXX  =  Sequential  number 

o  IVS  and  Testpit:  EllisStatIVSXXXXX.tem,  where 

■  XXXXX  =  Sequential  number 

•  TEMTADS 

o  Baekground:  bkgdxxxx.tem,  where 

■  XX  =  sequential  number 
o  Static  Data:  ttXXX.tem,  where 

■  XXX  =  sequential  number 
o  IVS:  ivsYYttsXX,  where 

■  YY  =  am  or  pm 

■  XX  =  sequential  number 
o  Testpit:  tpttsXX,  where 

■  XX  =  sequential  number 

■  YYYYMMDD  =  Year,  Month,  and  day. 

■  XXXXX  =  Sequential  number 

The  MetalMapper  and  TEMTADS  data  were  eollected  in  binary  .tern  files.  MetalMapper  data 
was  converted  to  a  .esv  file  using  the  TEM2CSV  program  and  then  imported  into  a  Geosoft 
database  (.gdb)  for  processing.  TEMTADS  data  was  exported  to  .csv  file  on  the  field  computer 
and  then  imported  into  a  Geosoft  database  (.gdb)  for  processing.  Geophysieal  data  is  contained 
in  Appendix  C  (see  separate  Addendum)  and  has  been  organized  into  raw  and  processed  data 
folders.  Eaeh  of  the  datasets  is  contained  in  a  single  .gdb  file.  Additional  information  on  the 
organization  of  the  data  is  contained  in  Appendix  C  (see  separate  Addendum). 

5.6  VALIDATION 

At  the  conclusion  of  data  collection  activities,  all  anomalies  on  the  master  dig  list  developed 
from  the  dynamic  MetalMapper  data  were  exeavated  by  ARC  ADIS’  UXO  Technieians,  who  met 
the  requirements  of  Technical  Paper- 18.  Eaeh  item  encountered  was  identified,  photographed, 
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its  depth  measured,  its  loeation  determined  using  em-level  RTK  DGPS,  and  the  item  removed. 
Intrusive  investigation  and  demolition  proeedures  followed  the  standard  operating  proeedures 
ineluded  in  the  approved  Demonstration  Plan.  The  intrusive  investigation  results  were 
doeumented  on  field  forms  and  transeribed  into  an  MS  Exeel  spreadsheet  that  was  provided  to 
the  ESTCP  Program  Offiee  and  is  ineluded  in  Appendix  D  (see  separate  Addendum)  of  this 
report. 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 

There  were  two  faeets  to  the  data  analysis  for  this  demonstration: 

•  Dynamie  data  was  pre-proeessed  and,  in  eonjunetion  with  the  ESTCP  Program  Office, 
target  lists  were  generated  from  the  MetalMapper  and  EM61-MK2  datasets  that  covered 
the  entire  site. 

•  Static,  cued  survey  data  was  processed  to  subtract  background  measurements  and  to 
apply  a  geolocation  (for  the  TEMTADS  only). 

Once  the  data  static  data  was  pre-processed,  ARCADIS  employed  physics-based  models  to 
extract  target  parameters  and  then  applied  a  library  matching  classification  algorithms  contained 
in  Oasis  Montaj’s  advanced  EiX- Analyze  Advanced  module  to  produce  a  ranked  anomaly  list. 

6.1  PREPROCESSING 

Data  was  preprocessed  using  instrument-specific  procedures.  These  preprocessed  data  was 
provided  for  use  in  target  parameter  extraction.  Below  are  the  instrument-specific  pre¬ 
processing  steps  that  were  employed  during  this  demonstration. 

6.1.1  EM61-MK2 

ARCADIS  performed  data  file  QC  review  of  and  correction  of  the  following: 

•  Transect  or  Grid  name  and  location,  and 

•  Eine  numbers,  survey  direction,  start  and  end  points. 

Additional  processing  of  the  EM61-MK2  data  was  conducted  in  Geosoft  Oasis  Montaj®  and 
included  applying  drift  corrections  {i.e.,  leveling  the  data)  and  latency  corrections.  Once  these 
corrections  were  applied,  the  data  processor  gridded  the  data  and  targets  were  selected. 

6.1.2  MetalMapper 

Dynamic  and  static  MetalMapper  data  was  pre-processed  by  converting  the  raw  .tern  file  to  the 
.CSV  file  using  Snyder  Geoscience,  Inc.’s  TEM2CSV  software  package.  TEM2CSV  was  also 
used  to  convert  the  GPS-supplied  latitude/longitude  data  to  UTM  coordinates,  correct  the  survey 
location  point  using  attitude  data  for  the  MetalMapper  platform  (heading,  pitch,  and  roll),  and 
remove  the  background  field  from  all  of  the  receiver  transients  (for  static  data  only). 

Dynamic  MetalMapper  data  was  then  leveled  in  Geosoft  Oasis  Montaj®.  No  latency  correction 
was  applied  because  there  was  no  apparent  latency  in  the  dynamic  dataset.  The  response  for 
time  gates  6  through  11  were  then  summed  and  average  for  the  inner  five  receivers.  The 
averaged  response  for  the  five  inner  receivers  was  then  gridded  for  target  selection. 

After  pre-processing  static  data  in  TEM2CSV,  the  .csv  file  was  then  imported  into  Geosoft  Oasis 
Montaj®  processing  environment  for  further  analysis. 
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6.1.3  TEMTADS  MP  2x2 


At  the  time  of  data  colleetion,  there  was  no  eommereially  available  software  to  convert  the  raw, 
dynamic  TEMTADS  data  in  .tern  format  to  a  .csv  file;  therefore,  SAIC  converted  the  files  and 
imported  them  to  an  Oasis  Montaj  geodatabase.  Once  in  Montaj,  channel  4  of  the  monostatic  Z 
response  {i.e.,  Z  transmit  and  Z  receive  coil)  was  multiplied  by  -1  to  make  the  response  positive 
and  then  leveled.  No  latency  correction  was  applied  because  there  was  no  apparent  latency  in 
the  dynamic  dataset.  Once  the  data  were  leveled,  the  leveled,  channel  4  was  gridded  for  target 
selection. 

The  static  TEMTADS  data  was  exported  to  .csv  files  on  the  field  computer.  In  preprocessing  on 
the  field  computer,  the  recorded  signals  were  normalized  by  the  peak  transmitter  current  to 
account  for  any  variation  in  the  transmitter  output.  The  .csv  files  were  imported  to  Geosoft  Oasis 
Montaj®  and  then  background  corrections  were  applied  prior  to  further  analysis,  which  is 
described  below.  The  background  response  was  subtracted  from  each  target  measurement  using 
data  collected  at  a  nearby  target-free  background  location.  The  background  measurements  were 
reviewed  for  variability  and  to  identify  outliers,  which  might  have  corresponded  to 
measurements  over  metallic  items. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Anomalies  were  selected  from  the  dynamic  DGM  survey  data  using  a  target  response-based 
procedure.  The  target  selection  threshold  for  each  advanced  EMI  sensor  was  based  on  the 
response  for  the  anticipated  Area  A  TOI,  which  corresponded  to  the  following: 

•  2.36  inch  rocket  down  to  18  inches  ('-'7.2  milliVolts  [mV]  on  EM61-MK2  channel  2) 

•  Hand  grenade  down  to  8  inches  (-'9.4  mV  on  EM61-MK2  channel  2) 

•  Aluminum  rifle  grenade  down  to  10  inches  (-'25.5  mV  on  EM61-MK2  channel  2). 

The  approximate  EM61-MK2  channel  2  response  for  the  first  two  TOI  listed  above  were 
estimated  using  NRL’s  Response  Calculator,  while  the  aluminum  rifle  grenade  response  was 
measured  in  the  field  at  the  test  pit.  The  final  target  selection  criteria  was  based  on  the  smallest 
Channel  2  response  of  the  2.36-inch  rocket  and  included  greater  than  a  50%  safety  factor  {i.e., 
the  final  selection  threshold  was  decreased  by  over  half  to  account  for  response  variations  due  to 
variations  in  instrument  height).  The  target  selection  threshold  for  the  dynamic  MetalMapper 
and  TEMTADS  data  was  based  on  a  comparison  of  the  measured  response  of  the  advanced  EMI 
sensors  to  the  EM6I-MK2  response  for  the  TOI  listed  above  at  the  TOI-specific  target  depths. 
The  final  targeting  thresholds  for  each  dataset  are  listed  below: 

•  EM61-MK2:  3.0  mV  channel  2 

•  MetalMapper:  3.0  mV  sum  channels  6-11  averaged  on  inner  5  Z  coils 

•  TEMTADS:  1  mV  on  channel  8 

Targets  were  selected  from  gridded  EMI  data  for  anomalies  with  responses  greater  than  those 
listed  above.  Figures  6-1  through  6-4  show  the  processed  dynamic  data  for  the  EM61-MK2, 
MetalMapper  (lines  spaced  0.5-m  apart),  MetalMapper  (lines  spaced  0.4-m  apart),  and  for  the 
TEMTADS,  respectively.  The  EM61-MK2  and  MetalMapper  datasets  collected  across  the  entire 
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Figure  6-1:  Dynamic  EM61-MK2  Data 
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Figure  6-2:  Dynamic  MetalMapper  Data  on  0.5-m  Line  Spacing 
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Figure  6-3:  Dynamic  MetalMapper  Data  on  0.4-m  Line  Spacing 
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Figure  6-4:  Partial  Dynamic  TEMTADS  Data  within  Target  Area 
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site  had  a  total  of  1,471  and  1,785  anomalies  respeetively.  Initially,  the  plan  was  to  use  the 
EM61-MK2  targets  to  determine  the  targets  at  whieh  statie  data  would  be  colleeted;  however, 
several  blind  ISOs  were  not  detected  within  the  0.6-m  search  radius  for  the  EM6I-MK2.  Both 
ISOs  were  detected,  but  the  anomalies  were  double-peaked  and  the  peaks  were  greater  than  0.6- 
m  from  the  center  of  the  seed  item.  Because  these  targets  failed  the  QC  performance  criteria,  the 
MetalMapper  data  collected  across  the  entire  site  was  used  instead  to  identify  targets  that  would 
be  collected  using  the  advanced  sensors  in  static  mode.  During  static  data  QC,  the  QC 
geophysicist  performed  an  inversion  of  the  static  data  and  evaluated  the  data  to  determine  if  there 
were  potentially  multiple  targets  at  the  target  location.  If  multiple  potential  TOI  were  identified 
at  a  target  location,  an  additional  target  was  added  to  the  target  list.  The  target  lists  were 
provided  to  the  ESTCP  Program  Office  during  the  field  effort.  Targets  EE-1  through  EE- 1785 
were  the  initial  MetalMapper  targets,  while  targets  with  numbers  greater  than  1,785  were  added 
during  the  static  data  QC.  Static  data  was  collected  over  the  initial  1,785  targets  as  follows: 

•  299  targets  collected  over  the  target  area  using  both  the  MetalMapper  and  the 

TEMTADS; 

•  740  targets  collected  over  the  southern  half  of  the  site  with  only  the  MetalMapper;  and 

•  746  targets  collected  over  the  northern  half  of  the  site  with  only  the  TEMTADS  (see 

Figure  6-5). 

Figure  6-6  shows  a  comparison  of  the  anomaly  response  for  a  small  portion  of  the  site  within  the 
target  area  for  the  four  different  dynamic  datasets:  a)  EM61-MK2  data  with  0.5-m  line  spacing, 
b)  MetalMapper  data  with  0.5-m  line  spacing,  c)  MetalMapper  data  with  0.4-m  line  spacing,  and 
3)  TEMTADS  data  with  0.4-m  line  spacing.  The  vertical  and  horizontal  black  lines  are  northing 
and  easting  coordinates  and  are  5-m  apart  on  all  figures.  The  color  scales  for  this  figure  are  the 
same  as  for  Figures  6-1  through  6-4  for  the  respective  dataset.  A  qualitative  analysis  shows  that 
the  general  anomaly  shapes  for  the  EM61-MK2  and  MetalMapper  data  in  a)  and  b)  are  very 
similar.  This  is  to  be  expected  given  that  the  summing  of  channels  and  averaging  the  response 
over  the  inner  five  z  coils  made  the  MetalMapper  effectively  quite  similar  to  the  EM61-MK2. 
Decreasing  the  line  spacing  of  the  MetalMapper  to  0.4-m  (see  Figure  6-6  part  c)  appears  to  have 
removed  some  of  the  smoothing  seen  in  the  first  two  images,  while  slightly  increasing  the  ability 
to  identify  discrete  anomaly  peak.  The  TEMTADS  data  further  reduces  the  smoothing  and  is 
much  better  than  the  other  instruments  at  identifying  discrete  anomalies.  The  more  discrete 
anomalies  are  likely  due  to  not  averaging  the  response  across  multiple  sensors.  A  similar  type  of 
discretization  of  anomalies  should  be  accomplishable  using  the  split  cubes  on  the  MetalMapper 
{i.e.,  not  averaging  across  the  inner  5  Z  coils). 

6.3  PARAMETER  ESTIMATES 

ARCADIS  used  the  cued  MetalMapper  and  TEMTADS  data  to  perform  advanced  classification. 
Detection  survey  data  was  only  used  for  identification  of  targets  at  which  cued  survey  data  was 
collected  and  to  compare  the  datasets  in  a  qualitative  manner.  Each  selected  anomaly  was 
analyzed  using  the  multi-object  solver  algorithm  in  the  UX- Analyze  Advanced  module  within 
Oasis  Montaj.  Both  intrinsic  (size,  shape,  materials  properties)  and  extrinsic  (location,  depth, 
orientation)  parameters  were  estimated  in  these  analyses  and  a  list  of  the  relevant  target 
parameters  from  each  analysis  compiled. 
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Figure  6-5:  Static  Data  Collection  Summary. 
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Figure  6-6:  Dynamic  Data  Comparisons.  Comparison  of  dynamic  datasets  within  the  target  area. 
Black  lines,  representing  northing  and  easting  coordinates  are  5-m  apart,  a)  EM61-MK2  data 
collected  on  0.5-m  line  spacing,  b)  MetalMapper  data  collected  on  0.5-m  line  spacing,  c) 
MetalMapper  data  collected  on  0.4-m  line  spacing,  d)  TEMTADS  data  collected  on  0.4-m  line 
spacing. 


The  static  data  was  processed  using  UX-Analyze  Advanced  module  to  extract  the  three  principal 
axis  polarizability  curves  for  each  target.  For  the  static  data,  ARCADIS  then  matched  the 
polarizability  curves  for  each  target  to  a  library  of  polarizability  curves  to  classify  the  target  as 
either  TOI  or  non-TOI. 


6.4  CLASSIFIER  AND  TRAINING 

ARCADIS’  initial  classifier  for  both  advanced  EMI  sensors  involved  matching  the  measured 
polarizabilities  to  a  library  that  contained  TOI  from  previous  sites  and  the  site-specific  TOI 
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tested  in  the  test  pit.  The  size  and  shape  of  polarizabilities  (or  Ps)  were  matehed  to  the  known 
library  items  for  the  following  three  scenarios: 

•  3  component  target  classiflcation 

o  Size  -  Pi 
o  Shape  1  -  P2/  Pi 
o  Shape  1  -  Ps/  Pi 

•  2  component  target  classiflcation 

o  Size  -  Pi 
o  Shape  1  -  P2/  Pi 

•  1  component  target  classiflcation 

o  Size  -  Pi 

Based  on  this  initial  classification,  training  data  was  requested  from  the  program  office  for  49 
MetalMapper  and  51  TEMTADS  targets.  The  training  data  was  requested  to  verify  the  library 
matching  model  that  was  being  proposed  and  included  ground  truth  from  excavations  at  this  site. 
The  particular  training  data  that  was  requested  tended  towards  relatively  low  confidence  metrics 
(e.g.,  0.7  -  0.85)  to  aid  in  determining  the  appropriate  location  of  the  dig  threshold.  Of  the 
requested  training  data,  only  5  MetalMapper  and  3  TEMTADS  targets  were  actually  TOIs. 
Nearly  all  of  the  remaining  requested  training  were  munitions  debris  (MD);  primarily  2.36” 
Rocket  Weights  (see  Figure  6-7)  and  2.36”  Rocket  Motors  (see  Figure  6-8).  Both  of  these  MD 
items  have  similar  characteristics  to  the  TOI  at  the  site.  The  rocket  weights  are  very  similar  to 
the  small  ISOs  that  were  used  for  blind  seeding  and  the  rocket  motors  were  very  similar  in  size 
and  shape  to  actual  2.36”  rockets.  In  order  to  account  for  the  apparently  large  number  of  rocket 
weights  and  motors  at  the  site,  five  MD  items  from  both  the  MetalMapper  and  TEMTADS 
training  data  were  added  to  their  respective  libraries  and  all  targets  were  re-classified  for  the  3-, 
2-,  and  1 -component  matches. 
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Figure  6-7:  Example  2.36”  Rocket  Weight 


Figure  6-8:  Example  2.36”  Rocket  Motor 

6.5  CLASSIFICATION 
6.5.1  Initial  Classification 

ARCADIS  produced  one  ranked  anomaly  list  for  the  MetalMapper  library  matehing  approaeh 
and  two  ranked  anomaly  lists  for  the  TEMTADS  library  matching  approach.  The  library 
matehing  approaeh  for  both  sensors  included  matehing  to  both  TOI  and  the  MD  clutter  items 
added  to  the  library  that  are  diseussed  above.  The  final  classification  used  a  combined 
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confidence  metric  that  averaged  the  confidence  metric  for  the  best  fitting  TOI  (or  clutter  item  in 
the  library)  for  the  3-,  2-,  and  1-  component  matches.  The  best  fitting  TOI  (or  clutter  item)  for 
the  3-component  match  was  then  assigned  as  the  TOI  Type.  ARCADIS  then  classified  each 
anomaly  as  one  of  the  following  categories: 

•  Category  -1:  anomalies  for  which  training  labels  had  been  requested.  These  anomalies 
were  placed  at  the  top  of  the  dig  list. 

•  Category  0:  Anomalies  for  which  ARCADIS  was  not  able  to  extract  reliable  parameters. 

•  Category  1:  Anomalies  that  had  a  combined  metric  match  to  TOI  greater  than  0.85. 
These  anomalies  were  believed  to  have  a  high  likelihood  of  being  TOI. 

•  Category  2:  Anomalies  that  had  a  combined  metric  match  to  TOI  between  0.8  and  0.85. 
ARCADIS  was  unsure  whether  these  anomalies  were  TOI  or  non-TOI. 

•  Category  3:  Anomalies  that  had  a  combined  metric  match  to  TOI  less  than  0.8  or  a 
combined  metric  match  to  Clutter  greater  than  0.85.  These  anomalies  had  a  high 
likelihood  of  being  clutter  items  and/or  a  low  likelihood  of  being  TOI.  These  anomalies 
were  placed  below  the  dig  threshold. 

Figure  6-9  shows  a  size-decay  plot  for  the  best  3-component  matches  for  the  TEMTADS  data. 
Items  classified  as  one  of  the  MD  clutter  items  are  shown  in  green  squares.  Two  clusters  of 
clutter  items  are  seen:  one  near  the  top  and  one  in  the  central  region  of  the  size  decay  plot.  These 
clusters  are  circled  in  red  on  Figure  6-9.  The  central  clutter  cluster  has  considerable  overlap 
with  small  ISOs,  which  are  displayed  as  pink  circles.  As  discussed  above,  ARCADIS  attempted 
to  try  to  identify  anomalies  that  were  likely  to  be  due  to  clutter  MD  items  to  reduce  the  number 
of  digs  required  to  identify  all  of  the  TOI  on  the  site.  In  order  to  do  this,  ARCADIS  classified 
targets  that  had  a  confidence  metric  greater  than  0.85  and  a  best  fit  to  a  Clutter  Item  as  a  category 
3  anomaly.  One  ranked  dig  list  was  submitted  for  the  MetalMapper  data  and  two  ranked  dig  lists 
were  submitted  for  the  TEMTADS:  Dig  Eist  A  (for  aggressive)  and  Dig  Eist  C  (for 
conservative).  No  modifications  were  made  to  the  Dig  Eist  C;  however,  the  TEMTADS  Dig  Eist 
A  was  further  modified  to  attempt  to  remove  the  central  clutter  cluster  seen  in  Figure  6-9  from 
the  dig  list.  For  TEMTADS  Dig  Eist  C,  all  anomalies  within  the  central  cluster  were  re¬ 
classified  as  Category  3  and  placed  below  the  cut  line. 

Table  6-1  shows  an  example  of  how  ARCADIS  ordered  our  ranked  anomaly  list.  Category  -1, 
0,  1,  2  anomalies  were  placed  above  the  dig  threshold  in  descending  order,  while  Category  3 
anomalies  were  placed  below  the  dig  threshold.  The  first  items  on  each  ranked  anomaly  list 
were  those  targets  for  which  ground  truth  labels  were  requested  (training  data).  Following  this, 
anomalies  for  which  reliable  parameters  could  not  be  extracted  and  therefore  had  to  be  dug  were 
listed.  Next  were  the  items  that  ARCADIS  was  the  most  confident  are  “highly  likely”  to  be  TOI. 
The  items  were  ranked  according  to  decreasing  confidence  that  the  item  is  TOI.  Any  items  that 
ARCADIS  was  able  to  analyze,  but  were  not  able  to  classify  {i.e.,  Category  2  anomalies)  were 
placed  next  on  the  anomaly  list.  Finally,  all  Category  3  items  that  ARCADIS  was  confident 
were  not  TOI  were  ranked  by  their  confidence.  The  Category  3  anomalies  were  ordered  in 
descending  order  based  on  their  confidence  metric,  which  led  to  the  best  match  to  clutter  items 
being  the  highest  ranking  Category  3  anomaly  on  the  dig  list. 
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Figure  6-9:  Size-Decay  X-Y  Scatter  Plot  of  TEMTADS  Data.  Green  squares  represent  the  targets 
classified  as  clutter  items  {Le.,  2.36”  Rocket  Weights  or  Motors).  Two  clusters  of  clutter  items  are 
circled  in  red:  one  in  the  top  and  one  in  the  central  portion  of  the  size  decay  plot. 


Table  6-1:  Example  Ranked  Anomaly  List 


Target 

ID 

Category 

Dig 

Decision 
(l=Dig; 
0=Do 
Not  Dig) 

Type 

(mm) 

Confidence 

Metric 

Comment 

EL-24 

-1 

1 

127 

1 

Training  Data 

Training  Data 

EL-339 

-1 

1 

127 

0.65 

Training  Data 

EL- 17 

0 

1 

127 

9999 

Can't  Analyze 

EL-862 

0 

1 

127 

9999 

Can't  Analyze 

Target 

ID 

Category 

Dig 

Decision 
(l=Dig; 
0=Do 
Not  Dig) 

Type 

(mm) 

Confidence 

Metric 

Comment 

EL-625 

2 

1 

60 

0.85 

Cannot  Decide 

EL- 167 

2 

1 

60 

0.8 

Cannot  Decide 

EL-710 

3 

0 

0 

1 

High  Confidence  Match  to 
Clutter  Items 

EL- 1006 

3 

0 

0 

0.8 

High  Confidence  Match  to 
Clutter  Items 

EL-631 

3 

0 

0 

0.79 

Low  Confidence  Match  to 
TOI  and  Clutter  Items 

EL-978 

3 

0 

0 

0 

Low  Confidence  Match  to 
TOI  and  Clutter  Items 

Note; 

mm  -  millimeter 


In  addition,  ARCADIS  provided  an  assignment  to  the  ‘Type’  column,  indicating  the  specific 
type  of  munition  caliber  (z.e.,  75  mm).  The  final,  ranked  anomaly  lists  are  provided  in  Appendix 
E  (see  separate  Addendum). 

6,5,2  Feedback  Based  on  QC  Seed  Performance 

The  initial  ranked  anomaly  lists  were  analyzed  to  confirm  that  all  QC  seeds  had  been  classified 
correctly.  QC  seed  items  were  not  marked  to  be  dug  on  the  initial  dig  list;  therefore,  the  ESTCP 
Program  Office  provided  the  location  and  identity  of  the  misclassified  QC  seeds  as  additional  on¬ 
site  training  data.  ARCADIS  used  this  additional  information  to  modify  our  classification 
procedures  to  ensure  that  the  revised  methods  correctly  classify  the  seeds  missed  originally.  A 
second  stage  list  was  then  submitted  accompanied  by  a  Failure  Analysis  memo  outlining  the 
causes  of  the  misclassification,  the  revisions  made  in  classification  procedures,  and  a 
demonstration  that  the  revised  procedures  successfully  classified  the  missed  seeds.  Appendix  F 
(see  separate  Addendum)  contains  the  QC  Seed  Failure  Memos  for  the  MetalMapper  and 
TFMTADS  dig  lists.  Upon  acceptance  of  the  memos  by  the  FSTCP  Program  Office,  ARCADIS 
submitted  Final  Ranked  Dig  Fists  (see  Appendix  E  in  separate  Addendum)  to  the  Program 
Office  and  the  second  stage  list  was  scored  by  IDA. 

6,6  DATAPRODUCTS 
6,6,1  Ranked  Anomaly  Lists  and  Results 

As  discussed  above,  ARCADIS  submitted  a  total  of  three  ranked  anomaly  lists:  one  for  the 
MetalMapper  and  two  for  the  TFMTADS  data.  Upon  submittal  of  the  final  ranked  dig  lists,  IDA 
constructed  Receiver  Operating  Characteristic  (ROC)  curves  for  each  anomaly  list.  Figures  6- 
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10,  6-11,  and  6-12,  show  the  ROC  curves  for  the  MetalMapper,  TEMTADS  Dig  List  A,  and 
TEMTADS  Dig  List  C,  respectively. 
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Figure  6-10:  MetalMapper  ROC  curve  with  four  points  of  interest  indicated 
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Figure  6-11:  TEMTADS  Dig  List  A  ROC  curve  with  four  points  of  interest  indicated 
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Figure  6-12:  TEMTADS  Dig  List  C  ROC  curve  with  four  points  of  interest  indicated 


The  key  regions  to  interpret  the  ROC  curves  are: 

•  The  black  dashed  line  corresponds  to  the  targets  that  were  dug  for  training  data. 

•  The  solid  black  line  corresponds  to  the  targets  that  were  categorized  as  “can’t  extract 
reliable  parameters”  that  were  treated  as  potential  TOI  because  no  meaningful 
classification  could  be  done.  At  least  one  TOI  was  identified  in  this  part  of  the  curve  for 
each  of  the  three  ROC  curves. 

•  Targets  in  red,  yellow,  green,  correspond  to  Category  1,  2,  and  3  targets,  respectively. 

•  The  blue  dot  corresponds  to  the  dividing  point  between  TOI  and  not-TOI. 

•  The  orange  dot  corresponds  to  the  point  at  which  ARCADIS  detected  all  TOI. 

As  discussed  in  Section  3,0,  the  primary  performance  metric  was  the  point  at  which  the  ROC 
curve  reaches  100%  identification  of  TOI.  The  number  of  clutter  items  correctly  identified  at 
this  point  is  a  measure  of  the  savings  possible  for  each  method.  As  seen  in  Figures  6-10  through 
6-12,  ARCADIS  missed  two  TOI  on  the  MetalMapper  dig  list  and  one  TOI  on  each  of  the 
TEMTADS  dig  lists.  Section  7,0  contains  an  assessment  as  to  why  these  TOI  were  incorrectly 
classified  as  non-TOI.  As  a  secondary  metric,  the  number  of  items  before  point  B,  those  that 
could  not  be  reliably  analyzed,  will  be  assessed. 
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6.6.2  Intrusive  Investigation  Results 

As  discussed  in  Section  5,6,  ARCADIS  UXO  technicians  dug  each  target  location  and  recorded 
the  type  of  item,  its  measured  depth,  its  location  determined  using  cm-level  RTK  DGPS,  if  the 
item  was  removed,  as  well  as  took  a  photograph  of  each  recovered  item.  Intrusive  investigation 
results  are  included  in  Appendix  D  (see  separate  Addendum)  of  this  report  and  the  photographs 
of  recovered  items  are  on  file  with  the  ESTCP  Program  Office. 

6.6.3  Raw  and  Processed  Dynamic  and  Static  Data 

As  discussed  above,  the  dynamic  and  static  QC,  IVS,  and  production  area  data  are  included  in 
Appendix  C  (see  separate  Addendum)  of  this  report  and  are  on  file  with  the  ESTCP  Program 
Office. 


53 


7.0  PERFORMANCE  ASSESSMENT 

Performance  objectives  were  established  in  the  demonstration  plan  to  evaluate  the  quality  of  data 
collected  as  part  of  this  demonstration.  These  performance  objectives  were  hrst  discussed  in 
Section  3.0  of  this  report.  This  section  documents  the  results  and  evaluates  the  data  quality  and 
whether  the  performance  metrics  were  met.  Table  7-1  shows  the  performance  results  for  each  of 
the  performance  objectives,  which  include  the  following: 

•  Data  Collection  Objectives 

o  Spatial  Coverage  in  Detection  Survey 
o  Along-line  Measurement  Spacing 
o  Detection  of  ah  TOI 
o  Repeatability  of  IVS  Measurements 
o  Production  Rate 
o  Cued  Interrogation  of  Anomalies 

•  Analysis  and  Classification  Objectives 

o  Maximize  Correct  Classification  of  TOI 
o  Maximize  Correct  Classification  of  Non-TOI 
o  Specification  of  No-Dig  Threshold 
o  Minimize  Number  of  Anomalies  that  Can’t  be  Analyzed 
o  Correct  Estimation  of  Target  Parameters. 

The  following  sub-sections  present  the  results  of  the  performance  assessment. 

7.1  DATA  COLLECTION  OBJECTIVES 
7.1,1  Objective:  Spatial  Coverage  for  Detection  Results 

As  discussed  above,  four  dynamic  detection  surveys  were  conducted  during  this  demonstration: 
EM61-MK2  and  MetalMapper  surveys  across  the  entire  5 -acre  site,  and  MetalMapper  and 
TEMTADS  surveys  across  the  partial  investigation  area  centered  on  the  target  area.  Success  for 
the  partial  Spatial  Coverage  metric  was  that  98%  coverage  was  achieved.  The  percent  coverage 
was  calculated  for  each  of  the  datasets  using  the  calculate  footprint  coverage  algorithm  in 
Geosoft  Oasis  Montaj’s  UX-Detect  module.  Table  7-1  shows  the  results  for  each  of  the  datasets 
and  Figures  7-1  through  7-4  show  the  results  for  the  full  EM61-MK2,  full  MetalMapper,  partial 
MetalMapper,  and  partial  TEMTADS  datasets,  respectively.  Due  to  the  large  number  of 
receivers  on  the  MetalMapper,  the  line  paths  completely  obscure  the  coverage  footprint; 
therefore  Figure  7-2a  shows  the  lines  paths  and  Figure  7-2b  shows  just  the  footprint  coverage. 
Ah  of  the  datasets  pass  the  coverage  metric,  with  the  exception  of  the  partial  TEMTADS  dataset. 
During  data  collection  with  the  TEMTADS,  the  PVC  handle  broke  from  the  platform  and  had  to 
be  repaired.  The  handle  repair  took  several  days  to  collect  the  appropriate  parts  and  perform  and 
the  project  team  determined  that  completing  the  data  collection  on  the  western  side  of  the  site 
and  filling  in  the  missing  line  was  not  required  to  perform  the  comparison  with  the  other 
dynamic  datasets. 
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Table  7-1:  Performance  Results 


Performance  Objective 

Metric 

Data  Required 

Success  Criteria 

EM61-MK2  Results 

MetalMapper  Results 

TEMTADS  Results 

Data  Collection  Objectives 

Spatial  coverage  in  detection 
survey 

Extended  footprint  coverage 

Mapped  survey  data 

98%  coverage 

Pass:  99.99%  coverage 

Pass:  0.5m  line  spacing:  100%  coverage 
0.4m  line  spacing:  100%  coverage 

Fail:  95.3%  coverage  achieved  before 
handle  broke  on  instrument.  After 
repair  of  the  instrument,  did  not 
recollect  missing  lines  since  sufficient 
data  existed  for  comparisons  to  EM6 1  - 
MK2  and  MetalMapper  data. 

Along-line  measurement 
spacing 

Point-to-point  spacing  from  data  set 

Mapped  survey  data 

98%  of  data  <  0. 15  m 

Pass:  100%  <  0.15m  down-line  spacing 

Pass:  0.5m  line  spacing  and  0.4m  line 
spacing  datasets  had  99.95%  and  99.99% 

<  0. 15m  down-line  spacing 

Fail:  96.5%  <  0.15m  down-line 
spacing. 

Detection  of  all  TOl 

Percent  detected  of  seeded  items 

Eocation  of  seeded  items; 
Anomaly  list 

1 00%  of  seeded  items 
detected  with  0.6  m 
halo 

Fail:  2  seed  items  failed.  Both  seed  items 
were  detected  as  double-peaked  anomalies. 
Targets  were  selected  at  the  two  peaks,  but 
each  target  was  more  than  0.6  m  away  from 
the  center  of  the  seed  item. 

0.5m  line  spacing:  pass 

0.4m  line  spacing:  NA;  intentionally 
collected  data  on  a  small  portion  of  the  site 

NA;  intentionally  collected  data  on  a 
small  portion  of  the  site. 

Repeatability  of  IVS 
measurements 

Amplitude  of  EM  anomaly 

Twice-daily  IVS  survey 
data 

Detection:  Amplitude 
within  25% 

Pass:  All  amplitudes  within  25%  of  expected 
amplitude. 

Pass:  All  amplitudes  within  25%  of 
expected  amplitude 

Not  Assessed 

Detection:  Down-track 
location  ±25  cm 

Pass:  All  offsets  less  than  25  cm 

Pass:  All  offsets  less  than  25  cm 

Not  Assessed 

Amplitude  of  polarizabilities 

Cued:  Polarizabilities 
±10% 

NA 

Pass:  All  polarizabilities  within  ±/-10%  of 
expected 

Pass:  All  polarizabilities  within  ±/-10% 
of  expected 

Production  rate 

Number  of  acres  of  data  collection  per 
day 

Eog  of  field  work  and 
data  analysis  time 
accurate  to  1 5  minutes 

Survey:  1  acres  per 
day 

Pass:  1.15  -1.83  acres/day  (average  of  1.5 
acres/day) 

Pass:  1.28-1.61  acres/day  (average  of  1.45 
acres/day)  for  0.5-m  line  spacing;  0.75 
acres/day  for  0.4-m  line  spacing 

Fail:  0.8  acres/day 

Time  required  to  analyze  each  target 

Analysis  time:  <5 
minutes  per  target 

NA 

Pass:  Analysis  time  =  3.8  minutes  per 
target 

Pass:  Analysis  time  =  3.8  minutes  per 
target 

Cued  interrogation  of 
anomalies 

Instrument  position 

Cued  survey  data 

1 00%  of  anomalies 
where  the  center  of  the 
instrument  is 
positioned  within  40 
cm  of  actual  target 
location 

NA 

Fail:  All  anomalies  pass  except  EL-406, 
which  had  an  offset  between  the 
MetalMapper  array  and  the  seed  item  of 
0.51  meters 

Fail:  All  anomalies  pass  except  EL-143 
had  an  offset  between  the  TEMTADS 
array  center  and  the  seed  item  of  0.49 
meters. 
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Performance  Objective 

Metric 

Data  Required 

Success  Criteria 

EM61-MK2  Results 

MetalMapper  Results 

TEMTADS  Results 

Analysis  and  Classification  Objectives 

Maximize  correct  classification 
ofTOl 

Number  of  TOl  retained 

Ranked  anomaly  lists; 
Scoring  reports  from  IDA 

Approach  correctly 
classifies  all  TOl 

NA 

Fail:  Anomalies  El-332  and  El-406 
incorrectly  classified  as  non-TOl 

Fail:  Anomaly  EE-1440  incorrectly 
classified  as  non-TOl 

Maximize  correct  classification 
of  non-TOl 

Number  of  false  alarms  eliminated 

•  Ranked  anomaly  lists 

•  Scoring  reports  from 
IDA 

Reduction  of  clutter 
digs  required  by  >75% 
while  retaining  all  TOl 

NA 

Fail:  submitted  dig  list  only  reduced  clutter 
digs  by  59.2%.  Actual  number  of  digs  to 
retain  all  TOl  resulted  in  approximately 
17.2%  reduction  in  clutter  digs. 

Fail:  submitted  dig  lists  A  and  C 
reduced  clutter  digs  by  approximately 
55%  and  50%,  respectively.  Actual 
number  of  digs  to  retain  all  TOl 
resulted  in  approximately  28%  and 
28%  reduction  in  clutter  digs. 

Specification  of  no-dig 
threshold 

Probability  of  correct  classification  of 
TOl  and  number  of  false  alarms  at 
demonstrator  operating  point 

•  Demonstrator  - 
specified  threshold 

•  Scoring  reports  from 
IDA 

Threshold  specified  by 
the  demonstrator  to 
achieve  criteria  above 

NA 

Category  1 :  >  0.85  confidence  metric  match  to  TOl 

Category  2:  <0.85  and  >0.8  confidence  metric  match  to  TOl 

Category  3  (non-TOl):  <  0.8  match  to  TOl  and  >  0.85  match  to  clutter  items  in 
library 

Minimize  number  of  anomalies 
that  cannot  be  analyzed 

Number  of  anomalies  that  must  be 
classified  as  “Cannot  Analyze” 

•  Demonstrator  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >  95%  of 
anomalies  on  the 
detection  list. 

NA 

Fail:  7.02%  of  targets  did  not  have  reliable 
target  parameters 

Fail:  6.82%  of  targets  did  not  have 
reliable  target  parameters 

Correct  estimation  of  target 
parameters 

Accuracy  of  estimated  target  locations 
for  seed  items 

•  Target  parameters 

•  Results  of  intrusive 
investigation 

X,Y  <15cm(lG) 

Z  <  1 0  cm  ( 1  a) 

NA 

Fail:  seven  seed  item  targets  failed  the 
horizontal  metric  and  three  failed  the 
vertical  metric. 

Fail:  four  seed  item  targets  failed  the 
horizontal  metric  and  three  failed  the 
vertical  metric. 

56 


MAP 

Footprint  Coverage 


LEGEND 


Area  covered:  20398.36 
Total  area:20400,00 
Percentage  covered;  99.99  % 
Instrument  foot  print;  1 .00  m 


Seale  1:1350 


Grid  North:  OW 

Mag  North:  O'D'O” 

Client: 

Project: 

Contractor 

Created  by: 

1  Verified  by: 

Date: 

File:  Camp aii»AII Grid»uee... 

Page  number 

Approved: 

Figure  7-1:  EM61-MK2  Data  Coverage 


57 


MAP 

Footprint  Coverage 


LEGEND 


Area  covered:  20399.98 
Total  area;20400.00 
Percentage  covered:  100.00 
Instrument  foot  print:  1 .00  m 


Scale  1;13S0 


N 

Grid  North;  0*0’0'‘ 

Meg  North;  0*0’0'’ 

Client: 

Project: 

Contractor: 

Craatad  by: 

1  Verified  by: 

Date: 

File;  SMC_Dynamk:_MM_half-me... 

Page  number: 

Approved: 

Figure  7-2a:  Full  MetalMapper  Dataset  Data  Coverage  -  Data  Line  Paths  only 
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Figure  7-2b:  Full  MetalMapper  Dataset  Data  Coverage  without  Line  Paths 
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Figure  7-3:  Partial  MetalMapper  Dataset  Data  Coverage 
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7.1.2  Objective:  Along-Line  Measurement  Spacing  Results 

The  reliability  of  the  survey  data  depends  on  the  density  of  coverage  of  the  site.  This  objective 
concerns  the  ability  of  the  instrument  operator  to  collect  data  with  acceptable  along-line 
measurement  spacing.  The  along  line  measurement  spacing  was  calculated  using  Geosoft  Oasis 
Montaj®  to  determine  if  greater  than  98%  of  the  data  had  along  line  measurement  spacings  less 
than  15  cm.  Figures  7-5  through  7-8  show  the  along  line  data  results  for  the  EM61-MK2,  full 
MetalMapper  dataset,  partial  MetalMapper  dataset,  and  the  TEMTADS  dataset,  respectively.  As 
see  on  the  figures  and  on  Table  7-1,  all  of  the  datasets  met  the  performance  objective  with  the 
exception  of  the  TEMTADS  dataset.  3.5%  of  the  TEMTADS  dataset  exceeded  the  along  line 
measurement  spacing;  however,  over  99.8%  of  the  dataset  had  a  data  spacing  less  than  20  cm.  It 
should  also  be  noted  that  typical  along  line  data  spacing  requirements  are  typically  on  the  order 
of  25  cm.  The  source  of  the  exceedances  are  that  the  instrument  was  being  pushed  too  quickly. 

7.1.3  Objective:  Detection  of  All  TOI  Results 

The  EM61-MK2  and  MetalMapper  datasets  collected  across  the  entire  site  had  a  total  of  1,471 
and  1,785  anomalies  identified  within  them,  respectively.  The  ESTCP  Program  Office  evaluated 
the  performance  of  the  dynamic  detection  of  all  TOIs  so  that  the  seed  items  remained  blind  to 
ARC  ADIS  geophysicists.  The  initial  demonstration  plan  was  to  use  the  EM61-MK2  targets  to 
determine  the  targets  at  which  static  data  would  be  collected;  however,  several  blind  ISOs  were 
not  detected  within  the  0.6-m  search  radius  for  the  EM61-MK2.  Both  ISOs  were  detected,  but 
the  anomalies  were  double-peaked  and  the  peaks  were  greater  than  0.6-m  from  the  center  of  the 
seed  item.  Because  these  targets  failed  the  QC  performance  criteria,  the  ESTCP  Program  Office 
evaluated  the  targets  from  the  full  MetalMapper  dataset  and  determined  that  the  MetalMapper 
dataset  successfully  located  all  of  the  seed  items  within  the  0.6-m  halo  and  was,  therefore,  used 
as  the  basis  for  the  cued  target  locations. 

7.1.4  Objective:  Repeatability  of  IVS  Measurements 

IVS  measurements  were  collected  twice  a  day  (one  in  the  morning  and  one  at  night)  to  confirm 
proper  instrument  functionality  and  sensor  system  performance.  The  metrics  for  this  objective 
were  the  amplitude  and  down-track  position  of  the  maxima  for  the  advanced  EMI  systems  in 
survey  mode  and  the  standard  deviation  of  the  polarizabilities  for  the  advanced  systems  in  cued 
mode  obtained  from  each  of  the  twice-daily  surveys  of  the  IVS. 

The  dynamic  data  from  the  EM61-MK2  and  MetalMapper  in  dynamic  were  met  if  the  measured 
amplitudes  for  each  object  are  within  25%  of  the  mean  and  the  down- track  position  of  the 
anomaly  peak  is  within  25  cm  of  the  known  location.  The  objective  will  be  considered  met  for 
the  advanced  systems  in  cued  mode  if  the  standard  deviation  of  the  estimated  polarizabilities  is 
within  10%  of  the  mean.  All  IVS  data  passed  these  metrics  and  the  IVS  data  is  contained  in 
Appendix  C  (see  separate  Addendum). 

7.1.5  Objective:  Production  Rate  Results 

The  metric  for  this  objective  is  the  number  of  acres  of  data  collection  per  day  and  the  time 
required  to  analyze  each  target  of  cued  survey  data.  The  objective  will  be  considered  to  be  met  if 
the  production  rate  of  1  acre  per  day  is  met  during  dynamic  data  acquisition  and  if  the  production 
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Figure  7-5:  EM61-MK2  Along-line  Measurement  Spacing 
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Figure  7-6:  Full  MetalMapper  Dataset  Along-line  Measurement  Spacing 
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Figure  7-7:  Partial  MetalMapper  Dataset  Along-line  Measurement  Spacing 
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Figure  7-8:  Partial  TEMTADS  Dataset  Along-line  Measurement  Spacing 
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rate  is  less  than  5  minutes  per  target  on  average  for  the  cued  anomaly  interrogation.  The 
achieved  dynamic  production  rates  were: 

•  EM61-MK2:  1.15  -  1.83  acres/day  (average  of  1.5  acres/day) 

•  Full  MetalMapper  Dataset:  1.28  -  1.61  acres/day  (average  of  1.45  acres/day) 

•  Partial  MetalMapper  Dataset:  0.75  acres/day 

•  Partial  TEMTADS  Dataset:  0.8  acres/day 

Both  of  the  full  datasets  exceeded  the  1  acre/day  performance  metric  and  therefore,  pass.  The 
partial  datasets  had  a  smaller  line  spacing  {i.e.,  40cm  vs.  50  cm)  and,  therefore,  took  a  longer 
time  to  collect.  Both  partial  datasets  fail  the  metric  established  in  the  demonstration  plan; 
however,  the  established  metric  was  designed  for  the  larger  line  spacing  and  not  for  the  partial 
datasets  that  were  added  at  a  later  time. 

The  static  data  analysis  time  was  calculated  based  on  the  amount  of  time  spent  to  QC  and 
analyze  the  static  data.  Both  the  MetalMapper  and  TEMTADS  data  took  approximately  3.8 
minutes  per  target,  which  is  below  the  5  minutes  per  target  metric  and  therefore,  both  pass. 

7.1.6  Objective:  Cued  Interrogation  of  Anomaly  Results 

The  reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data  collection 
in  relation  to  the  actual  anomaly  location.  The  objective  was  considered  to  be  met  if  the  center  of 
the  instrument  is  positioned  within  40  cm  of  the  actual  anomaly  location  for  100%  of  the  cued 
anomalies.  The  objective  was  evaluated  by  measuring  the  distance  of  the  seed  item  center 
location  vs.  the  location  of  the  center  of  the  cued  sensor  used  to  collect  data  over  the  seed  items. 
The  center  of  the  cued  sensors  were  within  the  0.4-m  metric  for  all  seed  items  except  for  one  for 
each  of  the  sensors.  The  center  of  the  MetalMapper  was  0.5 1-m  away  from  the  center  of  the 
seed  item  at  target  EE-406,  while  the  center  of  the  TEMTADS  was  0.49-m  away  from  the  center 
of  the  seed  item  at  target  EE- 143. 

7.2  ANALYSIS  AND  CLASSIFICATION  OBJECTIVES 
7.2.1  Objective:  Maximize  Correct  Classification  of  TOI  Results 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  correct 
classification  of  TOI.  The  metric  for  this  objective  is  the  number  of  items  on  the  anomaly  list  for 
a  particular  sensor  that  can  be  correctly  classified  as  TOI  by  each  classification  approach. 

ARCADIS  prepared  ranked  anomaly  lists  for  the  cued  MetalMapper  and  TEMTADS  datasets 
and  IDA  personnel  used  their  scoring  algorithms  to  assess  the  results.  The  objective  was 
considered  to  be  met  if  all  of  the  TOI  were  correctly  labeled  as  TOI  on  the  ranked  anomaly  list. 

Figures  6-10,  6-II,  and  6-12  show  the  ROC  curves  for  ARCADIS’  MetalMapper,  TEMTADS 
Dig  Eist  A,  and  TEMTADS  Dig  Eist  C,  respectively.  As  seen  on  these  figures,  ARCADIS 
incorrectly  classified  two  TOI  on  the  MetalMapper  (EE-406  and  EE-332)  and  one  TOI  on  both 
of  the  TEMTADS  dig  lists  (El-1440).  The  following  sub-sections  discussed  the  reason  for  the 
failure  to  correctly  classify  each  of  these  targets. 
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7.2. 1.1,  EL-332  Assessment 


As  shown  on  Figure  6-10,  ARCADIS  incorrectly  classified  EL-332  as  non-TOI.  Figure  7-9 
shows  one  realization  of  the  fit  results  for  EL-332,  while  Figure  7-10  shows  the  two  best  library 
matehes  for  the  3-,  2-,  and  1-eomponent  matehes.  As  seen  on  Figure  7-10,  the  best  two  matches 
for  the  three  matches  are  all  to  Clutter  items  in  the  library,  except  for  the  seeond  best  match  for 
the  2-component  mateh,  whieh  that  has  a  fit  metric  of  0.9434  to  an  ISO  from  Camp  Butner. 
Further  analysis  of  this  target  indicates  that  it  had  high  fit  metrics  to  ISOs  for  all  three  fit  matches 
and  should  have  been  identified  as  a  TOI  based  on  the  high  fit  metrie. 

7.2. 1.2.  EL-406  Assessment 

As  shown  on  Figure  6-10,  ARCADIS  incorreetly  classified  EL-406  as  non-TOI.  Figure  7-11 
shows  one  realization  of  the  fit  results  for  target  EL-406  and  Figure  7-12  shows  one  realization 
of  the  fit  results  to  target  EL-417,  whieh  is  closely  loeated  to  target  EL-406.  Target  A  at  eaeh 
target  loeation  have  fit  loeations  that  are  within  0.06  meters  of  each  other  and  are  the  same 
subsurface  item,  whieh  is  the  loeation  where  the  TOI  was  found.  While  the  Target  A  loeation  had 
a  relatively  good  fit  to  the  seed  item,  the  fit  at  EL-406  Target  B,  had  a  higher  metric  match  and 
was  the  target  used  to  classify  the  target  location.  Target  EL-406  had  a  fit  metric  of  0.76  to  a 
2.36-inch  rocket,  while  EL-417  was  correetly  elassified  as  a  small  ISO  and  was  placed  on  the  dig 
list  as  a  category  2  anomaly.  Although  ARACADIS  put  EL-406  below  the  dig  threshold,  the 
seed  item  would  have  been  found  beeause  the  dig  team  would  have  reeovered  the  seed  item 
while  digging  target  EL-417. 

7.2. 1.3.  EL-1440  Assessment 

As  shown  on  Figures  6-11  and  6-12,  ARCADIS  incorrectly  classified  target  EL-1440  as  non- 
TOI  on  both  TEMTADS  datasets.  Figures  7-13  and  7-14  show  the  fit  results  and  the  best  two 
matches  to  target  EL-1440  for  the  3-,  2-,  and  1-  component  matches,  respeetively.  Target  A  on 
Figure  17-13  has  the  best  fit  to  TOI  and  clutter  in  the  library  and  is  the  target  that  was  fit  to  the 
library  items  shown  in  Figure  17-14.  EL-1440  had  a  combined  fit  metric  of  0.9  and  its  best 
matehes  for  the  3-eomponent  and  2-compoent  fits  were  to  elutter  items.  The  polarization  curves 
shown  on  Figure  17-13  do  not  exhibit  normal  deeay  {i.e.,  betas  deereasing  as  a  function  of  time). 
Instead,  the  Target  A  P2  curve  shows  a  relatively  fiat  decay  over  the  middle  time  gates.  P3  also 
quickly  goes  into  the  noise,  whieh  is  seen  by  the  dashed  lines  starting  around  0.7  milliseconds. 
The  poor  decays  for  the  secondary  and  tertiary  betas  suggest  that  this  target  had  a  low  amplitude 
and  the  algorithm  had  difficulty  extracting  useful  polarization  parameters  throughout  the  decay 
curve.  In  addition,  the  non-normal  deeay  of  the  P2  curve  suggests  there  also  might  have  been  a 
poor  baekground  correetion  that  eaused  the  curve  to  remain  relatively  constant  over  time.  Based 
on  these  observations,  the  analyst  should  have  either  re-performed  the  baekground  correetion,  or 
only  used  the  1  -component  metric  match  in  the  classification. 

7.2.2  Objective:  Maximize  Correct  Classification  of  Non-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effeetiveness  of  the  classifieation 
approaeh.  This  objeetive  concerns  the  component  of  the  classifieation  problem  that  involves 
false  alarm  reduetion.  The  metrie  for  this  objective  is  the  number  of  items  on  the  sensor  dig  list 
that  can  be  correetly  classified  as  non-TOI  by  each  classification  approach.  The  objective  was 
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Figure  7-9:  EL-332  Fit  Results 
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Figure  7-10:  Best  2  Library  Matches  for  3-,  2-,  and  1-Component  Matches  to  Target  332 
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Figure  7-11:  EL-406  Fit  Results 
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Figure  7-12:  EL-417  Fit  Results 
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Figure  7-13:  EL-1440  Fit  Results 
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Figure  7-14:  Best  2  Library  Matches  for  3-,  2-,  and  1-Component  Matches  to  Target  EL-1440 
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considered  to  be  met  if  more  than  75%  of  the  non-TOI  items  were  correctly  labeled  as  non-TOI 
while  retaining  all  of  the  TOI  on  the  dig  list.  As  discussed  above,  neither  dig  list  eorreetly 
identified  all  TOI  and  therefore  this  performance  metric  was  not  met.  Furthermore,  if  all  TOI 
had  been  identified  with  the  submitted  dig  list,  the  dig  reduction  would  still  not  have  been  met. 
Table  7-2  lists  the  percent  reduction  in  the  number  of  clutter  digs  if  all  TOI  had  been  found  at 
the  dig  threshold  and  the  actual  percent  reduction  of  clutter  items  at  the  point  where  all  TOI 
items  were  found.  It  should  be  noted  that  the  actual  percent  reductions  were  negatively  impacted 
by  plaeing  the  best  fits  to  clutter  items  immediately  below  the  dig  threshold,  whieh  had  the  effect 
of  pushing  the  missed  TOI  further  down  the  dig  list. 

Table  7-2:  Reduction  of  Clutter  Digs 


Dig  List 

Pereent  Reduction  of 
Clutter  Digs  if  all  TOI 
found  at  Dig  Threshold 

Actual  Percent  Reduction 
of  Clutter  Items  at  Point 
of  Einding  all  TOI 

MetalMapper 

59% 

17% 

TEMTADS  Dig  List  A 

55% 

28% 

TEMTADS  Dig  List  C 

50% 

28% 

Objective:  Specification  of 

No-Dig  Threshold  Results 

In  a  retrospective  analysis  as  was  performed  in  this  demonstration,  it  is  possible  to  tell  the  true 
classifieation  eapabilities  of  a  classification  procedure  based  solely  on  the  ranked  anomaly  list 
submitted  by  ARC  ADIS.  In  a  real-world  seenario,  all  targets  may  not  be  dug  so  the  suecess  of 
the  approach  will  depend  on  the  ability  of  an  analyst  to  accurately  specify  their  dig/no-dig 
threshold.  As  discussed  above,  the  No  Dig  Threshold  was  that  anomalies  that  had  less  than  a  0.8 
match  to  TOI  or  greater  than  a  0.85  mateh  to  clutter  items  in  the  library  were  plaeed  below  the 
dig  threshold.  The  objeetive  was  not  met  since  less  than  75%  of  the  non-TOI  items  were 
correctly  labeled  as  non-TOI  while  retaining  all  of  the  TOI  at  the  ARCADIS-speeified  threshold. 

7.2,4  Objective:  Minimize  Number  of  Anomalies  that  Can’t  be  Analyzed  Results 

Anomalies  for  which  reliable  parameters  eannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  plaeed  in  the  dig  eategory  and  reduce  the  effectiveness  of 
the  classification  process.  The  ARCADIS  ranked  anomaly  lists  includes  those  anomalies  for 
which  parameters  could  not  be  reliably  estimated  as  Category  0  anomalies.  The  objeetive  was 
considered  to  be  met  if  reliable  parameters  were  estimated  for  >  95%  of  the  anomalies  on  the 
ranked  anomaly  list.  Table  7-3  shows  the  pereent  of  anomalies  that  were  plaeed  in  the  “Can’t 

Table  7-3:  Can’t  Analyze  Anomalies 


Dig  List 

Number  of 
Targets 

Number  of  Can't 
Analyze  Targets 

Percent  of  Can't 
Analyze  Targets 

MetalMapper 

1003 

71 

7.08% 

TEMTADS  Dig  List  A 

1012 

69 

6.82% 

TEMTADS  Dig  List  C 

1012 

69 

6.82% 
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Analyze”  category  for  each  ranked  anomaly  list.  The  percent  of  can’t  analyzed  targets  are  6.82% 
and  7.02%  for  the  TEMTADS  and  MetalMapper  dig  lists,  respectively.  This  performance  metric 
was  not  met  for  either  sensor. 


7.2,5  Objective:  Correct  Estimation  of  Target  Parameters 

This  objective  involves  the  accuracy  of  the  target  parameters  that  are  estimated  in  the  first  phase 
of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately.  Accuracy  of  estimation  of  target  parameters  is  the  metric  for  this  objective. 
ARC  ADIS  compared  the  estimated  parameters  for  the  seed  items.  The  objective  was  considered 
to  met  if  the  estimated  X,  Y  locations  were  within  15  cm  (la)  and  the  estimated  depths  are 
within  10  cm  (la).  Eleven  of  the  seed  items  had  horizontal  offsets  greater  than  10  centimeters: 
seven  for  the  MetalMapper  and  four  for  the  TEMTADS  datasets.  Table  7-4  summarizes  the 
targets  that  had  offsets  between  the  seed  item  location  and  predicted  X  and  Y  locations  that  are 
greater  than  15  cm.  Most  are  only  still  are  within  20  cm  of  the  actual  target  location,  but  EE-406 
is  an  outlier.  EE-406  was  a  missed  TOI  and  was  discussed  above  in  Section  7,2, 1,2.  The 
incorrect  target  fit  results  were  submitted  on  the  ranked  dig  list,  which  caused  the  very  large 
offset.  It  should  be  noted  that  target  EE-417  was  close  to  EE-406  and  correctly  classified  the 
missed  seed  item;  therefore,  the  dig  team  still  would  have  found  the  item.  All  estimated  depth 
were  within  10  centimeters  of  the  measured  depth. 

Table  7-4:  Anomalies  with  Horizontal  Offsets  Greater  than  15  cm 


Instrument 

Anomaly 

Measure 

Measured 

Predicted 

Predicted 

Horizontal 

Easting* 

Northing* 

Easting* 

Northing* 

offset  (cm) 

EL-23 

720060.313 

4475815.42 

720060.269 

4475815.51 

10.17 

EL-108 

720087.428 

4475835.39 

720087.555 

4475835.28 

16.93 

EL-406 

720110.092 

4475842.88 

720110.671 

4475843.47 

82.81 

MetalMapper 

EL-879 

720123.585 

4475816.66 

720123.772 

4475816.6 

19.71 

EL-974 

720126.166 

4475817.05 

720126.238 

4475817.14 

11.47 

EL- 1552 

720161.979 

4475789.37 

720161.772 

4475789.35 

20.85 

EL- 1784 

720062.164 

4475800.14 

720062.013 

4475800.13 

15.13 

740 

720119.86 

4475880.33 

720119.744 

4475880.43 

15.35 

TEMTADS 

1226 

720135.576 

4475894.94 

720135.571 

4475895.1 

15.79 

1353 

720143.755 

4475822.59 

720143.737 

4475822.77 

18.09 

1543 

720160.651 

4475826.73 

720160.491 

4475826.72 

15.97 

Note: 

1-  Coordinates  are  provided  in  UTM  WGS84  Zone  15N  with  units  of  meters. 
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Six  seed  item  targets  had  depths  that  exeeeded  the  10  em  metrie:  three  MetalMapper  and  three 
TEMTADS  targets.  Table  7-5  summarizes  the  anomalies  that  had  measured  depths  that  were 
greater  than  10  em  off  from  the  measured  depth.  Most  only  slightly  exeeed  the  depth  differenee; 
however,  EL- 123  5  has  a  large  depth  difference  and  was  a  missed  QC  seed  and  is  further 
discussed  in  the  TEMTADS  failure  QC  seed  failure  memos  contained  in  Appendix  F  (see 
separate  Addendum). 

Table  7-5:  Anomalies  with  Depth  Differences  Greater  than  10  cm 


Instrument 

Anomaly 

Measured 

Depth 

(cm) 

Predicted 

Depth 

(cm) 

Depth 

Difference 

(cm) 

MetalMapper 

EL-108 

29 

16.82 

12.18 

EL-324 

18 

5.36 

12.64 

EL-406 

15 

26.77 

11.77 

TEMTADS 

939 

13 

-2.19 

15.19 

1235 

26 

65 

39.0 

1564 

21 

0.84 

20.16 
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8.0  COST  ASSESSMENT 


This  section  provides  cost  information  to  aid  in  helping  professional  involved  in  MR  project  to 
reasonably  estimate  costs  for  implementation  at  a  given  site.  This  section  is  broken  down  into 
sub-sections  that  discuss  the  cost  model,  cost  drivers,  and  cost  benefit  of  the  various  technologies 
employed  at  the  former  CEMR. 

8.1  COST  MODEL 

ARCADIS  tracked  costs  throughout  the  ESTCP  live  site  demonstration  at  the  former  CEMR  and 
developed  a  simple  cost  model  to  aid  professionals  in  the  field  to  understand  costing 
implications.  The  cost  model  reflects  all  cost  elements  that  would  be  required  for  implementing 
the  technologies  described  in  this  report,  as  well  as  the  planning,  and  reporting  requirements. 
Table  8-1  presents  the  cost  elements  for  implementing  the  CEMR  live  site  demonstration 
including  the  data  tracked  during  the  demonstration. 

Table  8-1:  Details  of  the  Costs  Tracked  by  ARCADIS 


Cost  Element 

Data  Tracked  During 
Demonstration 

Estimated  Costs 

Project  Planning 

•  Develop  project-specific 
plans: 

o  Demonstration  Plan 
o  Health  and  Safety  Plan 

•  General  pre-planning 
activities 

$26,746 

Mobilization 

•  Mobilization  of  geophysical 
and  UXO  teams  and 
equipment. 

$58,740.36 

Site  Preparation 

•  Site  Boundary  Surveys 

•  Blind  Seeding 

•  IVS  setup 

•  Test  Pit  Measurements 

•  Instrument  Training 

$39,067 

EM61-MK2  Data  Collection 

•  Data  collection  and 
processing  of  6  acres  (5 
acres  and  1  acre  of  re¬ 
collected  data) 

Total  Cost:  $43,430  (6  acres) 

Cost  per  Acre:  $7,238 

TEMTADS  Survey  Data 
Collection 

•  Dynamic  TEMTADS  data 
collection 

•  IVS  and  QC  tests 
(TEMTADS  rental  costs  are  not 
included) 

Total  Cost:  $5,761  (0.68  acres) 

Cost  per  Acre:  $8,472 

MetalMapper  Survey  Data 
Collection 

•  Dynamic  MetalMapper  data 
collection 

•  IVS  and  QC  tests 
(MetalMapper  rental  costs  are 
not  included,  but  tractor  costs 

Total  Cost:  $36,803 

Cost  per  Acre:  $6,479 
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Cost  Element 

Data  Tracked  During 
Demonstration 

Estimated  Costs 

are) 

TEMTADS  Cued  Data 
Collection  /  QC 

•  Target  reacquisition 

•  Cued  TEMTADS  data 
collection 

•  Data  Processing  and  QC 

•  IVS  and  QC  tests 
(TEMTADS  rental  costs  are  not 
included) 

Total  Cost:  $68,061 
(1087  anomalies) 

Cost  per  Anomaly:  $62.6 

MetalMapper  Cued  Data 
Collection/QC 

•  Cued  MetalMapper  data 
collection 

•  Data  Processing  and  QC 

•  IVS  and  QC  tests 
(MetalMapper  rental  costs  are 
not  included,  but  tractor  costs 
are) 

Total  Cost:  $46,219 
(1,195  anomalies) 

Cost  per  Anomaly:  $36.8 

Cued  Data  Analysis 

•  Target  parameter  extraction 

•  Advanced  anomaly 
classification 

Total  Cost:  $18,276 
(2,282  anomalies) 

Cost  per  Anomaly:  $8.1 

Validation  Digging 

•  Target  reaequisition 

•  Intrusive  investigation 

•  Intrusive  results  reporting 

•  Post-dig  anomaly  QC 

Total  Cost:  $300,516 
(1,785  anomalies) 

Cost  per  Anomaly:  $166.4 

Final  Report 

•  Develop  project-specific 

reports: 

•  Final  Report 

•  Final  Cost  and  Performance 
Report 

$20,000  (estimated) 

8.2  COST  DRIVERS 

In  general,  the  intrusive  investigation  eosts  are  the  largest  eost  drivers  on  MR  projeets. 

Additional  eost  drivers  inelude  the  following. 

•  Dynamic  data  collection:  The  EM-61-MK2  and  MetalMapper  dynamie  data  eosts  are 
very  similar  on  a  per/aere  basis;  however,  the  TEMTADS  is  slightly  more  eostly  due  to 
inereased  site  preparation  (e.g.,  setting  up  ropes  in  grids)  and  slower  produetion  rates  due 
to 

•  Static  data  collection:  In  general,  MetalMapper  data  eolleetion  rates  were  greater  than 
with  the  TEMTADS.  This  is  largely  due  to  two  data  positioning  needs;  the  TEMTADS 
requires  the  reaequisition  of  target  loeations  prior  to  eued  data  eolleetion,  while  the 
MetalMapper  does  not,  and  not  using  a  GPS  with  the  TEMTADS  likely  eaused  a  larger 
number  of  targets  to  be  re-eolleeted  due  to  poor  positioning. 

•  Intrusive  investigation  cost  savings:  The  eost  savings  assoeiated  with  a  redueed  number 
of  non-TOI  ean  lead  to  a  large  eost  savings  sinee  the  intrusive  investigation  eosts  are  the 
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largest  cost  drivers.  Although  the  total  number  of  non-TOI  reduced  during  this 
demonstration  did  not  meet  the  performance  objectives,  more  experienced  demonstrators 
have  shown  in  their  analysis  of  the  former  CEMR  advanced  EMI  data  that  the  reduction 
of  non-TOI  anomalies  by  75%  is  accomplishable  at  this  she. 

8.3  COST  BENEFIT 

The  primary  driver  for  implementing  advanced  classification  is  to  reduce  the  number  of  non-TOI 
targets  that  require  intrusive  investigation  and  thereby,  decrease  the  overall  costs  of  DoD’s 
MMRP  cost  to  complete.  Advanced  classification  has  been  shown  to  reduce  the  overall  number 
of  non-TOI  digs  by  60-90%  at  other  sites.  ARCADIS  successfully  reduced  the  number  of  non- 
TOI  digs  by  approximately  70%  at  Pole  Mountain,  while  retaining  all  of  the  TOI;  however,  we 
were  unsuccessful  at  reducing  the  number  of  non-TOI  digs  (while  retaining  all  of  the  TOI)  by 
more  than  20%  at  CEMR.  Other,  more  experienced  demonstrators,  however,  were  successful  at 
reducing  the  number  of  non-TOI  digs  by  75%  at  CEMR.  Tables  8-2  and  8-3  provide  cost- 
benefit  analyses  for  performing  advanced  classification  using  the  costs  listed  in  Table  8-1  for  the 
MetalMapper  and  TEMTADS,  respectively.  Static  costs  (e.g.,  planning,  reporting,  and 
mobilization)  and  costs  with  minimal  variability  (e.g.,  site  preparation)  are  not  included  in  the 
cost-benefit  analysis.  Each  scenario  is  compared  against  performing  a  traditional  MR  projects 
that  includes  an  EM61-MK2  survey  and  intrusive  investigation  of  all  anomalies.  Eor  the 
advanced  classification  costs,  it  is  assumed  that  the  respective  advanced  EMI  sensor  is  used  to 
collect  both  dynamic  survey  data  and  cued  data. 

Tables  8-2  and  8-3  each  contain  three  scenarios: 

•  Scenario  1:  Greatest  cost  savings  associated  within  reducing  the  number  of  targets 
requiring  intrusive  investigation  by  75%. 

o  MetalMapper  Cost  Savings:  44% 
o  TEMTADS  Cost  Savings:  28% 

•  Scenario  2:  Intermediate  cost  savings  associated  within  reducing  the  number  of  targets 
requiring  intrusive  investigation  by  50%. 

o  MetalMapper  Cost  Savings:  22% 
o  TEMTADS  Cost  Savings:  5% 

•  Scenario  3:  No  cost  savings  scenario.  This  is  the  scenario  at  which  there  is  no  advantage 
to  performing  advanced  classification.  The  percent  reduction  in  digs  for  this  scenario  for 
the  MetalMapper  and  TEMTADS  are  36.5  and  43,  respectively. 

The  MetalMapper  costs  shown  in  Table  8-1  were  lower  than  the  TEMTADS  costs,  and  therefore 
result  in  much  greater  cost  savings  than  the  TEMTADS. 
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Table  8-2:  MetalMapper  Cost  Evaluation 


Cost  Scenarios 

Cost  Element 

Cost 

per 

Unit 

Unit 

Quantity 

Traditional 
MR  Costs 

Classification 

Costs 

MetalMapper 
Scenario  1  - 
Greatest  Cost 
Savings:  75% 
Anomaly 
Reduction  with 
MetalMapper 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

MetalMapper 

Survey 

$6,479.0 

Acres 

5 

$0 

$32,395 

Cued  MetalMapper 

$36.8 

Anomaly 

2000 

$0 

$73,600 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  25%  of 
Anomalies 

$166.4 

Anomaly 

500 

$0 

$83,200 

Total: 

$368,990 

$205,395 

Cost  Savings  ($): 

$163,595 

Cost  Savings  (%) 

44% 

MetalMapper 
Scenario  2  - 
Intermediate 
Cost  Savings: 
50%  Anomaly 
Reduction  with 
MetalMapper 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

MetalMapper 

Survey 

$6,479.0 

Acres 

5 

$0 

$32,395 

Cued  MetalMapper 

$36.8 

Anomaly 

2000 

$0 

$73,600 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  50%  of 
Anomalies 

$166.4 

Anomaly 

1000 

$0 

$166,400 

Total: 

$368,990 

$288,595 

Cost  Savings  ($): 

$80,395 

Cost  Savings  (%) 

22% 

MetalMapper 
Scenario  3  -  No 
Cost  Savings: 
36.5%  Anomaly 
Reduction  with 
MetalMapper 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

MetalMapper 

Survey 

$6,479.0 

Acres 

5 

$0 

$32,395 

Cued  MetalMapper 

$36.8 

Anomaly 

2000 

$0 

$73,600 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  63.5%  of 
Anomalies 

$166.4 

Anomaly 

1480 

$0 

$246,272 

Total: 

$368,990 

$368,467 

Cost  Savings  ($): 

$523 

Cost  Savings  (%) 

0% 
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Table  8-3:  TEMTADS  Cost  Evaluation 


Cost  Scenarios 

Cost  Element 

Cost 

per 

Unit 

Unit 

Quantity 

Traditional 
MR  Costs 

Classiflcation 

Costs 

MetalMapper 
Scenario  1  - 
Greatest  Cost 
Savings:  75% 
Anomaly 
Reduction  with 
TEMTADS 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

TEMTADS  Survey 

$8,472.0 

Acres 

5 

$0 

$42,360 

Cued  TEMTADS 

$62.6 

Anomaly 

2000 

$0 

$125,200 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  25%  of 
Anomalies 

$166.4 

Anomaly 

500 

$0 

$83,200 

Total: 

$368,990 

$266,960 

Cost  Savings  ($): 

$102,030 

Cost  Savings  (%) 

28% 

MetalMapper 
Scenario  2  - 
Intermediate 
Cost  Savings: 
50%  Anomaly 
Reduction  with 
TEMTADS 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

TEMTADS  Survey 

$8,472.0 

Acres 

5 

$0 

$42,360 

Cued  TEMTADS 

$62.6 

Anomaly 

2000 

$0 

$125,200 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  50%  of 
Anomalies 

$166.4 

Anomaly 

1000 

$0 

$166,400 

Total: 

$368,990 

$350,160 

Cost  Savings  ($): 

$18,830 

Cost  Savings  (%) 

5% 

MetalMapper 
Scenario  3  -  No 
Cost  Savings: 
44%  Anomaly 
Reduction  with 
TEMTADS 

EM61-MK2  Survey 

$7,238.0 

Acres 

5 

$36,190 

$0 

TEMTADS  Survey 

$8,472.0 

Acres 

5 

$0 

$42,360 

Cued  TEMTADS 

$62.6 

Anomaly 

2000 

$0 

$125,200 

Classification 

$8.1 

Anomaly 

2000 

$0 

$16,200 

Dig  All  Anomalies 

$166.4 

Anomaly 

2000 

$332,800 

$0 

Dig  56%  of 
Anomalies 

$166.4 

Anomaly 

1120 

$0 

$186,368 

Total: 

$368,990 

$370,128 

Cost  Savings  ($): 

-$1,138 

Cost  Savings  (%) 

0% 
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9.0  IMPLEMENTATION  ISSUES 


In  general,  the  former  CEMR  was  a  relatively  easy  site  at  whieh  to  perform  advaneed 
classifieation.  The  site  terrain  was  flat  and  the  only  vegetation  that  was  of  coneem  were  corn 
stalks  that  rose  above  the  ground  surface  approximately  4  inches.  Neither  terrain  nor  vegetation 
proved  to  impact  implementation  once  the  com  stalks  were  cut  down  to  the  ground.  The  former 
CEMR  was  the  site  at  which  ARCADIS  collected  advanced  EMI  and  therefore,  there  were  some 
lessons  learned  regarding  the  implementation  of  the  technology,  which  are  summarized  below. 

•  MetalMapper  Data  Acquisition 

o  At  the  time  ARCADIS  collected  the  dynamic  MetalMapper  data,  there  was  no 
visual  method  for  determining  whether  the  transmit  coil  was  turned  on.  This  led 
to  inadvertently  collecting  data  without  the  transmit  coil  turned  on,  which  led  to 
extra  costs  to  re-collect  the  data. 

o  Prior  to  CEMR,  the  MetalMapper  had  typically  been  moved  via  a  tractor.  At 
CEMR  ,  the  MetalMapper  was  instead  mounted  on  a  forklift  that  provided  the 
following  benefits: 

■  The  forklift  had  a  heated  cab  in  which  the  field  technician  sat  during  data 
collection.  CEMR  was  conducted  in  winter,  so  the  heated  cab  improved 
working  conditions  and  helped  minimize  the  potential  for  cold-stress 
injuries. 

■  The  cab  provided  a  waterproof  location  for  the  placement  of  the  computer 
monitor,  which  was  not  mggedized  or  water  proof 

■  At  prior  live  site  demonstration,  MetalMapper  data  collection  had  been 
performed  using  a  tractor  with  a  monitor  placed  in  the  field  of  view  of  the 
tractor  driver  (see  Figure  9-1).  At  CEMR,  ARCADIS  placed  a  smaller 
monitor  within  the  heated  cab  that  increased  the  visibility  of  the 
equipment  operator  (see  Figure  9-2). 

•  TEMTADS  Data  Acquisition 

o  The  TEMTADS  was  difficult  to  push  through  the  project  site  during  dynamic  data 
acquisition  due  to  several  factors,  including: 

■  Eow  height  of  the  instrument,  which  caused  it  to  get  caught  on  com  stalks, 
even  after  the  com  stalks  were  cut  down  to  near  the  ground  surface. 

■  There  were  only  two  wheels  on  the  back  end  of  the  instmment  (note  that 
this  has  since  been  rectified  and  there  are  now  4  wheels  on  the 
TEMTADS). 

•  The  angle  of  the  handle  was  not  adjustable.  When  shorter  field  personnel  (e.g.,  5  ft  2-inch 
tall  personnel)  collected  data,  they  had  to  push  the  instmment  from  a  relatively  high 
position  that  required  extra  effort.  Eield  training  was  provided  by  the  instmment 
manufacturers  during  the  live  site  demonstration;  however,  this  training  was  primarily 
focused  on  data  collection  and  data  collection  QC  in  the  field  and  did  not  include  office 
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QC  procedures.  Although  both  SAIC  and  NAEVA  provided  some  as-needed  offiee  QC 
proeedures;  more  eodified  data  proeessing  and  evaluation  QC  proeedures  are 
reeommended. 


Figure  9-1:  Traditional  MetalMapper  Tractor  Operator  Field  of  View  (URS,  2013) 


Figure  9-2:  CEMR  MetalMapper  Equipment  Operator  Field  of  View 
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Appendix  A:  Points  of  Contact 


POINT  OF 
CONTACT 

ORGANIZATION 

Phone 

Pax 

e-mail 

Role  in 
Project 

Dr.  Jeff 
Marqusee 

ESTCP  Program  Office 
4800  Mark  Center  Drive, 
Suite  I7D08,  Alexandria, 
VA  22350-3605 

571-  372-6378  (V) 
i  effrev.marausee(2), osd.mil 

Director, 

ESTCP 

Dr.  Anne 
Andrews 

ESTCP  Program  Office 
4800  Mark  Center  Drive, 
Suite  17D08,  Alexandria, 
VA  22350-3605 

571-372-6379  (V) 
anne.andrews(2).osd.mil 

Deputy 

Director, 

ESTCP 

Dr.  Herb 
Nelson 

ESTCP  Program  Office 
4800  Mark  Center  Drive, 
Suite  17D08,  Alexandria, 
VA  22350-3605 

571-  372-6400  (V) 

202-215-4844  (C) 
herbert.nelson@osd.mil 

Program 
Manager,  MR 

Mr.  Daniel 
Ruedy 

HydroGeoLogic,  Inc. 

1 1 107  Sunset  Hills  Road, 
Suite  400 

Reston,  VA  20190 

703-736-4531  (V) 
druedv@hgl.com 

Program 
Assistant,  MR 

Dr.  Shelley 
Cazares 

Institute  for  Defense 
Analyses 

4850  Mark  Center  Drive 
Alexandria,  VA  223 1 1 

703-845-6792  (V) 
scazares@ida.org 

Performance 

Assessment 

Mr.  Steve 
Stacy 

ARCADIS 

3101  Wilson  Blvd,  Suite 
550 

Arlington,  VA  22201 

703-465-4234  (V) 

425-891-4507  (Cell) 

Steve.  Stacv@arcadis-us  .com 

ESTCP 

Principal 

Investigator 

Ms.  Elise 
Goggin 

USACE 

4820  University  Square 
ED-CS-G 

Huntsville ,  AE  35816 

256-895-1635  (V) 
Elise.M.Goggin@usace.armv.mil 

USACE 

Geophysicist 

Mr.  Jon  M. 
Ritterling,  PE 

URS  Corporation 
12120  Shamrock  Plaza, 
Suite  300 

Omaha,  NE  68154 

402-952-2502  (V) 
Jon.m.ritterling@urs.com 

USACE 
Contractor, 
Senior  Project 
Manager 
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POINT  OF 
CONTACT 

ORGANIZATION 

Phone 

Fax 

e-mail 

Role  in 
Project 

Mr.  Darrell 
Hall 

URS  Corporation 

12 120  Shamrock  Plaza, 
Suite  300 

Omaha,  NE  68154 

402-952-2682  (V) 
darrell.hall(S),urs.com 

USACE 

Contractor, 

Geophysicist 

Mr.  Clarence 
Smith 

Illinois  Environmental 
Protection  Agency 

217-524-1655  (V) 
clarence. smith(2),illinois. gov 

State 

Regulator 

Mr.  Paul  Lake 

Illinois  Environmental 
Protection  Agency 

217-785-7728  (V) 
naul .  lake(S)  Illinois  .gov 

State 

Regulator 
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Appendix  B:  GPS  Control  Point  Data  Sheets 


A  separate  Addendum  contains  information  regarding  Appendices  B  through  F  of  the  ESTCP 
Munitions  Response  Live  Site  Demonstration  Report  detailing  work  conducted  at  Former  Camp  Ellis 
by  Mr.  Steve  Stacy  of  ARCADIS-US,  Inc.  Appendices  B  through  F  contain  ancillary  data  and  results 
that  are  not  formatted  for  release  through  a  webpage  and  instructions  to  obtain  these  appendices  are 

available  in  the  Addendum. 
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Appendix  C:  Geophysical  Data 


A  separate  Addendum  contains  information  regarding  Appendices  B  through  F  of  the  ESTCP 
Munitions  Response  Live  Site  Demonstration  Report  detailing  work  conducted  at  Former  Camp  Ellis 
by  Mr.  Steve  Stacy  of  ARCADIS-US,  Inc.  Appendices  B  through  F  contain  ancillary  data  and  results 
that  are  not  formatted  for  release  through  a  webpage  and  instructions  to  obtain  these  appendices  are 

available  in  the  Addendum. 
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Appendix  D:  Intrusive  Results 


A  separate  Addendum  contains  information  regarding  Appendices  B  through  F  of  the  ESTCP 
Munitions  Response  Live  Site  Demonstration  Report  detailing  work  conducted  at  Former  Camp  Ellis 
by  Mr.  Steve  Stacy  of  ARCADIS-US,  Inc.  Appendices  B  through  F  contain  ancillary  data  and  results 
that  are  not  formatted  for  release  through  a  webpage  and  instructions  to  obtain  these  appendices  are 

available  in  the  Addendum. 
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Appendix  E:  Ranked  Anomaly  Lists 


A  separate  Addendum  contains  information  regarding  Appendices  B  through  F  of  the  ESTCP 
Munitions  Response  Live  Site  Demonstration  Report  detailing  work  conducted  at  Former  Camp  Ellis 
by  Mr.  Steve  Stacy  of  ARCADIS-US,  Inc.  Appendices  B  through  F  contain  ancillary  data  and  results 
that  are  not  formatted  for  release  through  a  webpage  and  instructions  to  obtain  these  appendices  are 

available  in  the  Addendum. 
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